2024-04-12T19:34:24.932 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-12T19:34:24.932 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533 --verbose --name vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/orig.config.yaml 2024-04-12T19:34:24.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T19:34:25.066 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533 branch: wip-vshankar-testing-20240411.061452 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} email: vshankar@redhat.com first_in_suite: false job_id: '7653533' kernel: client: branch: testing flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240411.061452 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 324baf8ddb91dea46cac50cb535c306115791355 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 324baf8ddb91dea46cac50cb535c306115791355 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240411.061452 sha1: 324baf8ddb91dea46cac50cb535c306115791355 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7105 sha1: 324baf8ddb91dea46cac50cb535c306115791355 sleep_before_teardown: 0 subset: 463/1024 suite: fs suite_branch: wip-vshankar-testing-20240411.061452 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 324baf8ddb91dea46cac50cb535c306115791355 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUEA8WnjKZ9paXPsDNdA3GYnTxMov/p6jCpaJpj6rVwsfiQF01MzE5aCFaFiMwm7KD0tEWLAYQRg8OjJZxcx2w= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCUDIFxY/GFxMBPu6HqD4URgoVn/Pp03fHbk3an3renx91zpeRxaJNCymGgVDBKijvsELAohvemX2LJoFj1EYao= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-12_03:11:38 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-12T19:34:25.066 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa; will attempt to use it 2024-04-12T19:34:25.067 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks 2024-04-12T19:34:25.067 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-12T19:34:25.068 INFO:teuthology.task.internal:Checking packages... 2024-04-12T19:34:25.093 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '324baf8ddb91dea46cac50cb535c306115791355' 2024-04-12T19:34:25.093 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T19:34:25.093 INFO:teuthology.packaging:ref: None 2024-04-12T19:34:25.093 INFO:teuthology.packaging:tag: None 2024-04-12T19:34:25.093 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240411.061452 2024-04-12T19:34:25.093 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T19:34:25.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240411.061452 2024-04-12T19:34:25.234 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2879.g324baf8d 2024-04-12T19:34:25.236 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-12T19:34:25.242 INFO:teuthology.task.internal:no buildpackages task found 2024-04-12T19:34:25.242 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-12T19:34:25.262 INFO:teuthology.task.internal:Saving configuration 2024-04-12T19:34:25.280 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-12T19:34:25.340 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T19:34:25.366 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 19:29:08.044209', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUEA8WnjKZ9paXPsDNdA3GYnTxMov/p6jCpaJpj6rVwsfiQF01MzE5aCFaFiMwm7KD0tEWLAYQRg8OjJZxcx2w='} 2024-04-12T19:34:25.387 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 19:29:08.042806', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCUDIFxY/GFxMBPu6HqD4URgoVn/Pp03fHbk3an3renx91zpeRxaJNCymGgVDBKijvsELAohvemX2LJoFj1EYao='} 2024-04-12T19:34:25.387 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-12T19:34:25.393 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-12T19:34:25.393 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-12T19:34:25.393 INFO:teuthology.run_tasks:Running task console_log... 2024-04-12T19:34:25.460 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9670ab1160>, signals=[15]) 2024-04-12T19:34:25.460 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-12T19:34:25.466 INFO:teuthology.task.internal:Opening connections... 2024-04-12T19:34:25.466 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:34:25.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:34:25.578 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:34:25.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:34:25.708 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-12T19:34:25.715 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-04-12T19:34:25.732 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-04-12T19:34:25.732 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-04-12T19:34:25.791 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-12T19:34:25.792 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-04-12T19:34:25.792 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T19:34:25.792 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-12T19:34:25.792 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T19:34:25.792 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-04-12T19:34:25.814 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-12T19:34:25.831 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-12T19:34:25.831 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-12T19:34:25.888 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T19:34:25.889 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-12T19:34:25.913 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-12T19:34:25.923 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-12T19:34:25.928 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-12T19:34:25.929 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-04-12T19:34:25.931 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-12T19:34:25.947 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-12T19:34:25.953 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-12T19:34:25.953 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-04-12T19:34:25.989 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-12T19:34:26.084 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-12T19:34:26.134 INFO:teuthology.run_tasks:Running task kernel... 2024-04-12T19:34:26.148 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-12T19:34:26.148 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-12T19:34:26.149 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-12T19:34:26.149 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T19:34:26.149 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T19:34:26.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:34:26.166 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-12T19:34:26.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:34:26.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-12T19:34:26.327 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:34:26.327 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-12T19:34:26.328 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-12T19:34:26.345 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-361.el9.x86_64 2024-04-12T19:34:26.345 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:34:26.345 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-12T19:34:26.345 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2024-04-12T19:34:26.500 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:34:26.501 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-12T19:34:26.501 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-12T19:34:26.519 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-361.el9.x86_64 2024-04-12T19:34:26.519 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:34:26.519 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-12T19:34:26.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum upgrade -y linux-firmware 2024-04-12T19:34:28.633 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-04-12T19:34:28.794 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-12T19:34:30.309 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.5 MB 00:00 2024-04-12T19:34:30.509 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.5 MB 00:00 2024-04-12T19:34:31.616 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-12T19:34:31.643 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-12T19:34:33.787 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-12T19:34:45.651 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 808 kB/s | 6.2 MB 00:07 2024-04-12T19:34:47.160 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-12T19:34:48.373 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-12T19:34:53.506 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 932 kB/s | 19 MB 00:20 2024-04-12T19:34:53.879 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-12T19:34:56.228 INFO:teuthology.orchestra.run.smithi032.stderr:Modular dependency problems: 2024-04-12T19:34:56.228 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-12T19:34:56.228 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 1: conflicting requests 2024-04-12T19:34:56.228 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 2: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 3: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 4: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 5: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 6: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 7: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 8: conflicting requests 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.229 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 9: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 10: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 11: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 12: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 13: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 14: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 15: conflicting requests 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.230 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 16: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 17: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 18: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 19: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 20: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 21: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 22: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 23: conflicting requests 2024-04-12T19:34:56.231 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 24: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 25: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 26: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 27: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 28: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 29: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 30: conflicting requests 2024-04-12T19:34:56.232 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 31: conflicting requests 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 32: conflicting requests 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 33: conflicting requests 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 34: conflicting requests 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 35: conflicting requests 2024-04-12T19:34:56.233 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 36: conflicting requests 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 37: conflicting requests 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 39: conflicting requests 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.234 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.235 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.236 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.237 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T19:34:56.750 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-12T19:34:56.758 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-04-12T19:34:56.759 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 1 Package 2024-04-12T19:34:56.759 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:34:56.759 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 376 M 2024-04-12T19:34:56.759 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-12T19:34:57.482 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-12T19:34:59.124 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-12T19:35:00.525 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-12T19:35:05.734 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-12T19:35:07.970 INFO:teuthology.orchestra.run.smithi169.stderr:Modular dependency problems: 2024-04-12T19:35:07.970 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-12T19:35:07.970 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 1: conflicting requests 2024-04-12T19:35:07.970 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 2: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 3: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 4: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 5: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 6: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 7: conflicting requests 2024-04-12T19:35:07.971 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 8: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 9: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 10: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 11: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 12: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 13: conflicting requests 2024-04-12T19:35:07.972 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 14: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 15: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 16: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 17: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 18: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 19: conflicting requests 2024-04-12T19:35:07.973 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 20: conflicting requests 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 21: conflicting requests 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 22: conflicting requests 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 23: conflicting requests 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 24: conflicting requests 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.974 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 25: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 26: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 27: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 28: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 29: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 30: conflicting requests 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.975 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 31: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 32: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 33: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 34: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 35: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 36: conflicting requests 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.976 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 37: conflicting requests 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 39: conflicting requests 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.977 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.981 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T19:35:07.981 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.981 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.981 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T19:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T19:35:08.527 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-12T19:35:08.537 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-12T19:35:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-04-12T19:35:08.537 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-12T19:35:08.537 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-12T19:35:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 1 Package 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 376 M 2024-04-12T19:35:08.538 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-12T19:36:01.116 INFO:teuthology.orchestra.run.smithi032.stdout:linux-firmware-20240219-143.el9.noarch.rpm 5.9 MB/s | 376 MB 01:04 2024-04-12T19:36:01.116 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-12T19:36:01.117 INFO:teuthology.orchestra.run.smithi032.stdout:Total 5.8 MB/s | 376 MB 01:04 2024-04-12T19:36:04.298 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-12T19:36:04.405 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-12T19:36:04.406 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-12T19:36:07.735 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-12T19:36:07.736 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-12T19:36:14.335 INFO:teuthology.orchestra.run.smithi169.stdout:linux-firmware-20240219-143.el9.noarch.rpm 5.7 MB/s | 376 MB 01:05 2024-04-12T19:36:14.336 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-12T19:36:14.336 INFO:teuthology.orchestra.run.smithi169.stdout:Total 5.7 MB/s | 376 MB 01:05 2024-04-12T19:36:15.180 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-12T19:36:15.525 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T19:36:15.614 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:16.641 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:16.641 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:36:17.056 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-12T19:36:17.373 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-12T19:36:17.374 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:36:17.381 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-12T19:36:17.536 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-12T19:36:17.536 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-12T19:36:17.741 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-12T19:36:17.741 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-12T19:36:17.744 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2024-04-12T19:36:20.714 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-12T19:36:20.715 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-12T19:36:27.944 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-12T19:36:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T19:36:28.388 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:29.368 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:29.368 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:36:29.825 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-12T19:36:30.132 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-12T19:36:30.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T19:36:30.352 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-12T19:36:30.352 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-12T19:36:30.354 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-04-12T19:36:51.009 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12 19:36:51 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-12T19:36:51.009 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2024-04-12 19:36:51-- 2024-04-12T19:36:51.009 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 21s 2024-04-12T19:36:51.009 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 341M in 20s (16.8 MB/s) 2024-04-12T19:36:51.012 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.1... 2024-04-12T19:36:51.013 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-12T19:36:53.061 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2024-04-12T19:36:53.204 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2024-04-12T19:36:53.283 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2024-04-12T19:37:01.218 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-12T19:37:18.003 INFO:teuthology.orchestra.run.smithi169.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-12T19:37:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12 19:37:19 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-12T19:37:19.114 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2024-04-12 19:37:19-- 2024-04-12T19:37:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 1m 1s 2024-04-12T19:37:19.115 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 341M in 1m 1s (5.60 MB/s) 2024-04-12T19:37:19.118 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-12T19:37:19.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-12T19:37:21.313 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying... ######################################## 2024-04-12T19:37:21.382 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing... ######################################## 2024-04-12T19:37:21.466 INFO:teuthology.orchestra.run.smithi032.stdout:Updating / installing... 2024-04-12T19:37:29.758 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-12T19:37:40.843 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-12T19:37:40.843 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-12T19:37:40.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-12T19:37:40.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-12T19:37:40.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-12T19:37:40.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-12T19:37:40.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-12T19:37:40.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-12T19:37:40.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-12T19:37:40.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-12T19:37:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-12T19:37:40.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-12T19:37:40.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-12T19:37:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-12T19:37:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-12T19:37:40.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-12T19:37:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-12T19:37:40.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-12T19:37:40.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-12T19:37:40.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-12T19:37:40.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-12T19:37:40.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-12T19:37:40.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-12T19:37:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-12T19:37:40.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-12T19:37:40.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-12T19:37:40.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-12T19:37:40.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-12T19:37:40.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-12T19:37:40.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-12T19:37:40.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-12T19:37:40.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-12T19:37:40.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-12T19:37:40.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-12T19:37:40.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-12T19:37:40.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-12T19:37:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-12T19:37:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-12T19:37:40.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-12T19:37:40.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-12T19:37:40.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-12T19:37:40.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-12T19:37:40.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-12T19:37:40.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-12T19:37:40.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-12T19:37:40.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-12T19:37:40.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-12T19:37:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-12T19:37:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-12T19:37:40.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-12T19:37:40.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-12T19:37:40.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-12T19:37:40.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-12T19:37:40.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-12T19:37:40.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-12T19:37:40.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-12T19:37:40.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-12T19:37:40.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-12T19:37:40.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-12T19:37:40.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-12T19:37:40.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-12T19:37:40.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-12T19:37:40.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-12T19:37:40.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-12T19:37:40.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-12T19:37:40.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-12T19:37:40.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-12T19:37:40.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-12T19:37:40.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-12T19:37:40.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-12T19:37:40.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-12T19:37:40.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-12T19:37:40.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-12T19:37:40.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-12T19:37:40.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-12T19:37:40.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-12T19:37:40.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-12T19:37:40.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-12T19:37:40.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-12T19:37:40.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-12T19:37:40.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-12T19:37:40.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-12T19:37:41.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-12T19:37:41.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-12T19:37:41.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-12T19:37:41.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-12T19:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-12T19:37:41.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-12T19:37:41.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-12T19:37:41.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-12T19:37:41.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-12T19:37:41.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-12T19:37:41.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-12T19:37:41.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-12T19:37:41.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-12T19:37:41.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-12T19:37:41.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-12T19:37:41.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-12T19:37:41.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-12T19:37:41.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-12T19:37:41.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-12T19:37:41.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-12T19:37:41.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-12T19:37:41.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-12T19:37:41.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-12T19:37:41.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-12T19:37:41.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-12T19:37:41.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-12T19:37:41.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-12T19:37:41.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-12T19:37:41.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-12T19:37:41.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-12T19:37:41.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-12T19:37:41.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-12T19:37:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-12T19:37:41.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-12T19:37:41.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-12T19:37:41.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-12T19:37:41.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-12T19:37:41.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-12T19:37:41.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-12T19:37:41.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-12T19:37:41.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-12T19:37:41.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-12T19:37:41.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-12T19:37:41.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-12T19:37:41.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-12T19:37:41.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-12T19:37:41.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-12T19:37:41.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-12T19:37:41.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-12T19:37:41.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-12T19:37:41.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-12T19:37:41.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-12T19:37:41.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-12T19:37:41.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-12T19:37:41.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-12T19:37:41.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-12T19:37:41.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-12T19:37:41.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-12T19:37:41.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-12T19:37:41.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-12T19:37:41.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-12T19:37:41.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T19:37:41.065 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:41.066 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:for file in vmlinuz System.map config; do 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout: fi 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:done 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-12T19:37:41.089 INFO:teuthology.orchestra.run.smithi169.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:41.090 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-12T19:37:41.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-04-12T19:37:41.174 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-04-12T19:37:41.174 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 68.el9 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8264254 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-04-12T19:37:41.175 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:37:41.176 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T19:37:41.177 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T19:37:41.177 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:41.177 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T19:37:41.818 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-04-12T19:37:43.268 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-12T19:37:43.302 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-04-12T19:37:43.304 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-12T19:37:43.318 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.aCaSCea8rJ 2024-04-12T19:37:43.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aCaSCea8rJ 2024-04-12T19:37:43.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.aCaSCea8rJ 2024-04-12T19:37:43.526 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.aCaSCea8rJ is 6KB 2024-04-12T19:37:43.576 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.aCaSCea8rJ 2024-04-12T19:37:43.592 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T19:37:43.662 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-12T19:37:43.662 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-12T19:37:43.662 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-12T19:37:43.662 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-12T19:37:43.663 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:37:43.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-04-12T19:37:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-12T19:38:08.253 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-12T19:38:08.253 DEBUG:teuthology.orchestra.run.smithi032:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-12T19:38:08.338 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-12T19:38:08.339 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-12T19:38:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-12T19:38:08.341 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-12T19:38:08.342 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-12T19:38:08.343 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-12T19:38:08.344 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-12T19:38:08.345 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-12T19:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-12T19:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-12T19:38:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-12T19:38:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-12T19:38:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-12T19:38:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-12T19:38:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-12T19:38:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-12T19:38:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-12T19:38:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-12T19:38:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-12T19:38:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-12T19:38:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-12T19:38:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-12T19:38:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-12T19:38:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-12T19:38:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-12T19:38:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-12T19:38:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-12T19:38:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-12T19:38:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-12T19:38:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-12T19:38:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-12T19:38:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-12T19:38:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-12T19:38:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-12T19:38:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-12T19:38:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-12T19:38:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-12T19:38:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-12T19:38:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-12T19:38:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-12T19:38:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-12T19:38:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-12T19:38:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-12T19:38:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-12T19:38:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-12T19:38:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-12T19:38:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-12T19:38:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-12T19:38:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-12T19:38:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-12T19:38:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-12T19:38:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-12T19:38:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-12T19:38:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-12T19:38:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-12T19:38:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-12T19:38:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-12T19:38:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-12T19:38:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-12T19:38:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-12T19:38:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-12T19:38:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-12T19:38:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-12T19:38:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-12T19:38:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-12T19:38:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-12T19:38:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-12T19:38:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-12T19:38:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-12T19:38:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-12T19:38:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-12T19:38:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-12T19:38:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-12T19:38:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-12T19:38:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-12T19:38:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-12T19:38:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-12T19:38:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-12T19:38:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-12T19:38:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-12T19:38:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-12T19:38:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-12T19:38:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-12T19:38:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-12T19:38:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-12T19:38:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-12T19:38:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-12T19:38:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-12T19:38:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-12T19:38:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-12T19:38:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-12T19:38:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-12T19:38:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-12T19:38:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-12T19:38:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-12T19:38:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-12T19:38:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-12T19:38:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-12T19:38:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-12T19:38:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-12T19:38:08.438 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-12T19:38:08.439 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-12T19:38:08.440 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-12T19:38:08.441 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-12T19:38:08.442 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-12T19:38:08.443 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-12T19:38:08.444 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-12T19:38:08.445 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-12T19:38:08.446 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-12T19:38:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-12T19:38:08.448 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-12T19:38:08.449 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-12T19:38:08.450 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-12T19:38:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-12T19:38:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-12T19:38:08.453 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-12T19:38:08.454 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-12T19:38:08.455 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-12T19:38:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-12T19:38:08.457 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-12T19:38:08.458 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-12T19:38:08.459 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-12T19:38:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-12T19:38:08.461 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-12T19:38:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-12T19:38:08.463 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-12T19:38:08.464 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-12T19:38:08.465 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-12T19:38:08.466 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-12T19:38:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-12T19:38:08.468 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-12T19:38:08.469 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-12T19:38:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-12T19:38:08.471 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-12T19:38:08.472 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-12T19:38:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-12T19:38:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T19:38:08.478 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.478 DEBUG:teuthology.orchestra.run.smithi032:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:postinstall scriptlet (using /bin/sh): 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:for file in vmlinuz System.map config; do 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: fi 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-12T19:38:08.502 INFO:teuthology.orchestra.run.smithi032.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:fi 2024-04-12T19:38:08.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.06 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 68.el9 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2024-04-12T19:38:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Group : Unspecified 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 8264254 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : builder@centos.org 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T19:38:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T19:38:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T19:38:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2024-04-12T19:38:08.597 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:38:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T19:38:08.598 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T19:38:08.598 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:08.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T19:38:09.203 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-04-12T19:38:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-12T19:38:10.691 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-04-12T19:38:10.694 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-12T19:38:10.709 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.UyxNFQAtZG 2024-04-12T19:38:10.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UyxNFQAtZG 2024-04-12T19:38:10.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.UyxNFQAtZG 2024-04-12T19:38:10.920 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.UyxNFQAtZG is 6KB 2024-04-12T19:38:10.969 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.UyxNFQAtZG 2024-04-12T19:38:10.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T19:38:11.051 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-12T19:38:11.052 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-12T19:38:11.052 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-12T19:38:11.052 INFO:teuthology.orchestra.run.smithi032.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-12T19:38:11.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:38:11.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-04-12T19:38:13.840 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-12T19:38:13.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:38:13.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:38:32.351 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-12T19:38:41.226 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-12T19:38:41.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:38:41.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:38:41.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:38:41.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:38:44.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-12T19:38:56.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:38:56.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:38:59.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-12T19:39:08.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:39:08.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:39:27.135 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-12T19:39:39.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:39:39.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:39:41.114 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-12T19:39:41.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:39:41.944 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-12T19:39:41.944 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-12T19:39:41.985 INFO:teuthology.orchestra.run.smithi032.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:39:41.985 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:39:41.985 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T19:39:41.985 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T19:39:42.986 DEBUG:teuthology.parallel:result is None 2024-04-12T19:39:56.429 DEBUG:teuthology.orchestra.remote:timed out 2024-04-12T19:40:11.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:40:11.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:40:11.898 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-12T19:40:12.606 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:40:12.606 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-12T19:40:12.607 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-12T19:40:12.659 INFO:teuthology.orchestra.run.smithi169.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:40:12.659 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T19:40:12.659 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T19:40:12.659 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T19:40:13.660 DEBUG:teuthology.parallel:result is None 2024-04-12T19:40:13.661 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-12T19:40:13.667 INFO:teuthology.task.internal:Creating test directory... 2024-04-12T19:40:13.668 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T19:40:13.677 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T19:40:13.734 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-12T19:40:13.744 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-12T19:40:13.750 INFO:teuthology.task.internal:Creating archive directory... 2024-04-12T19:40:13.750 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T19:40:13.757 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T19:40:13.818 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-12T19:40:13.825 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-12T19:40:13.825 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T19:40:13.831 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T19:40:13.966 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T19:40:14.002 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T19:40:14.033 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T19:40:14.077 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T19:40:14.082 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-12T19:40:14.089 INFO:teuthology.task.internal:Configuring sudo... 2024-04-12T19:40:14.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T19:40:14.094 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T19:40:14.192 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-12T19:40:14.200 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-12T19:40:14.201 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T19:40:14.208 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T19:40:14.259 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T19:40:14.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T19:40:14.457 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T19:40:14.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T19:40:14.631 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:40:14.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T19:40:14.711 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T19:40:14.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T19:40:14.898 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T19:40:14.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T19:40:15.068 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:40:15.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T19:40:15.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-04-12T19:40:15.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-12T19:40:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T19:40:15.346 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T19:40:15.830 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-12T19:40:15.838 INFO:teuthology.task.internal:Starting timer... 2024-04-12T19:40:15.839 INFO:teuthology.run_tasks:Running task pcp... 2024-04-12T19:40:15.848 INFO:teuthology.run_tasks:Running task selinux... 2024-04-12T19:40:15.857 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-12T19:40:15.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-04-12T19:40:15.990 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-04-12T19:40:15.994 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-12T19:40:16.119 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-12T19:40:16.125 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-12T19:40:16.125 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-04-12T19:40:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-04-12T19:40:16.185 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-12T19:40:16.239 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-12T19:40:16.240 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-12T19:40:16.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T19:40:16.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:40:16.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T19:40:16.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:40:16.411 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-12T19:40:16.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-04-12T19:40:16.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-12T19:40:16.612 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-12T19:40:16.621 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-12T19:40:16.626 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-12T19:40:17.272 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-12T19:40:17.289 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-12T19:40:17.291 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-12T19:46:53.504 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-12T19:46:53.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:46:53.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:46:53.661 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-12T19:46:53.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-12T19:46:53.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:46:53.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T19:46:53.976 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-12T19:46:54.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-12T19:46:54.137 INFO:teuthology.run_tasks:Running task clock... 2024-04-12T19:46:54.148 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-12T19:46:54.148 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T19:46:54.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T19:46:54.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T19:46:54.156 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T19:46:54.255 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T19:46:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T19:46:54.308 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T19:46:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T19:46:54.401 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-12T19:46:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-04-12T19:46:54.443 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-12T19:46:54.470 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-04-12T19:46:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T19:46:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T19:46:54.552 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T19:46:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T19:46:54.690 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-12T19:46:54.699 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T19:46:54.699 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-12T19:46:54.699 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.699 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.699 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.700 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.729 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.739 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:46:54.741 INFO:teuthology.run_tasks:Running task install... 2024-04-12T19:46:54.750 DEBUG:teuthology.task.install:project ceph 2024-04-12T19:46:54.750 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}} 2024-04-12T19:46:54.750 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'} 2024-04-12T19:46:54.750 INFO:teuthology.task.install:Using flavor: default 2024-04-12T19:46:54.756 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-12T19:46:54.756 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-12T19:46:54.757 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'tag': None, 'wait_for_package': False} 2024-04-12T19:46:54.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T19:46:54.759 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'tag': None, 'wait_for_package': False} 2024-04-12T19:46:54.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T19:46:55.075 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/ 2024-04-12T19:46:55.075 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2879.g324baf8d 2024-04-12T19:46:55.079 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/ 2024-04-12T19:46:55.079 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2879.g324baf8d 2024-04-12T19:46:55.189 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T19:46:55.189 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:46:55.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T19:46:55.267 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240411.061452/324baf8ddb91dea46cac50cb535c306115791355/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T19:46:55.268 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:46:55.268 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T19:46:55.298 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-12T19:46:55.298 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/324baf8ddb91dea46cac50cb535c306115791355/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T19:46:55.368 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-12T19:46:55.369 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/324baf8ddb91dea46cac50cb535c306115791355/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T19:46:55.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T19:46:55.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T19:46:55.583 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T19:46:55.643 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T19:46:55.681 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-04-12T19:46:55.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-04-12T19:46:55.728 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-12T19:46:55.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-12T19:46:56.071 INFO:teuthology.orchestra.run.smithi032.stdout:74 files removed 2024-04-12T19:46:56.086 INFO:teuthology.orchestra.run.smithi169.stdout:74 files removed 2024-04-12T19:46:56.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-12T19:46:56.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-12T19:46:56.975 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 237 kB/s | 83 kB 00:00 2024-04-12T19:46:57.087 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-04-12T19:46:57.526 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-04-12T19:46:57.536 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-04-12T19:46:58.062 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-12T19:46:58.135 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-12T19:46:58.766 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-04-12T19:46:58.872 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-12T19:47:00.625 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.5 MB 00:00 2024-04-12T19:47:00.706 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.5 MB 00:00 2024-04-12T19:47:01.860 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-12T19:47:02.737 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-12T19:47:03.978 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-12T19:47:05.628 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-12T19:47:08.721 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.2 MB 00:00 2024-04-12T19:47:10.305 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-04-12T19:47:11.023 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-04-12T19:47:12.249 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-12T19:47:12.816 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-12T19:47:18.365 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-12T19:47:19.108 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr:Modular dependency problems: 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 1: conflicting requests 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 2: conflicting requests 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 3: conflicting requests 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.020 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 4: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 5: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 6: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 7: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 8: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 9: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 10: conflicting requests 2024-04-12T19:47:21.021 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 11: conflicting requests 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 12: conflicting requests 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 13: conflicting requests 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 14: conflicting requests 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 15: conflicting requests 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.022 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 16: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 17: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 18: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 19: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 20: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 21: conflicting requests 2024-04-12T19:47:21.023 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 22: conflicting requests 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 23: conflicting requests 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 24: conflicting requests 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 25: conflicting requests 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 26: conflicting requests 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.024 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 27: conflicting requests 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 28: conflicting requests 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 29: conflicting requests 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 30: conflicting requests 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 31: conflicting requests 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.025 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 32: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 33: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 34: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 35: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 36: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 37: conflicting requests 2024-04-12T19:47:21.026 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 39: conflicting requests 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.027 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.028 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.029 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.030 INFO:teuthology.orchestra.run.smithi169.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T19:47:21.681 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-12T19:47:21.683 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-12T19:47:21.704 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-12T19:47:21.706 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-12T19:47:21.707 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-12T19:47:21.709 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-12T19:47:21.711 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-12T19:47:21.715 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-12T19:47:21.716 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-12T19:47:21.818 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:========================================================================================= 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:========================================================================================= 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 6.6 k 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 5.4 M 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 889 k 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 145 k 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 1.5 M 2024-04-12T19:47:21.839 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 142 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 3.5 M 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 7.4 M 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 49 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 10 M 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 49 M 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 264 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 762 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 224 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 27 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 18 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 32 k 2024-04-12T19:47:21.840 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 746 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 127 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 161 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 320 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 297 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 99 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 85 k 2024-04-12T19:47:21.841 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.1 M 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 171 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-12T19:47:21.842 INFO:teuthology.orchestra.run.smithi169.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.4 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.1 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-12T19:47:21.843 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 21 M 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 34 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 2.4 M 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 249 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 4.7 M 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 17 M 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 16 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 25 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-12T19:47:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 164 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 500 k 2024-04-12T19:47:21.845 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 5.2 M 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-12T19:47:21.846 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 45 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 135 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-12T19:47:21.847 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-12T19:47:21.848 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-12T19:47:21.849 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-12T19:47:21.850 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-12T19:47:21.851 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:========================================================================================= 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:Install 150 Packages 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 24 Packages 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 229 M 2024-04-12T19:47:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-12T19:47:23.049 INFO:teuthology.orchestra.run.smithi169.stdout:(1/174): ceph-19.0.0-2879.g324baf8d.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-12T19:47:23.375 INFO:teuthology.orchestra.run.smithi169.stdout:(2/174): ceph-fuse-19.0.0-2879.g324baf8d.el9.x8 2.7 MB/s | 889 kB 00:00 2024-04-12T19:47:23.466 INFO:teuthology.orchestra.run.smithi169.stdout:(3/174): ceph-immutable-object-cache-19.0.0-287 1.6 MB/s | 145 kB 00:00 2024-04-12T19:47:23.675 INFO:teuthology.orchestra.run.smithi169.stdout:(4/174): ceph-base-19.0.0-2879.g324baf8d.el9.x8 5.8 MB/s | 5.4 MB 00:00 2024-04-12T19:47:23.776 INFO:teuthology.orchestra.run.smithi169.stdout:(5/174): ceph-mds-19.0.0-2879.g324baf8d.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-04-12T19:47:23.943 INFO:teuthology.orchestra.run.smithi169.stdout:(6/174): ceph-mgr-19.0.0-2879.g324baf8d.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-12T19:47:24.268 INFO:teuthology.orchestra.run.smithi169.stdout:(7/174): ceph-mon-19.0.0-2879.g324baf8d.el9.x86 9.6 MB/s | 4.7 MB 00:00 2024-04-12T19:47:25.376 INFO:teuthology.orchestra.run.smithi169.stdout:(8/174): ceph-radosgw-19.0.0-2879.g324baf8d.el9 9.0 MB/s | 10 MB 00:01 2024-04-12T19:47:25.425 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-12T19:47:25.660 INFO:teuthology.orchestra.run.smithi169.stdout:(9/174): ceph-common-19.0.0-2879.g324baf8d.el9. 7.1 MB/s | 21 MB 00:02 2024-04-12T19:47:25.685 INFO:teuthology.orchestra.run.smithi169.stdout:(10/174): ceph-selinux-19.0.0-2879.g324baf8d.el 82 kB/s | 25 kB 00:00 2024-04-12T19:47:25.752 INFO:teuthology.orchestra.run.smithi169.stdout:(11/174): cephfs-mirror-19.0.0-2879.g324baf8d.e 3.3 MB/s | 224 kB 00:00 2024-04-12T19:47:26.002 INFO:teuthology.orchestra.run.smithi169.stdout:(12/174): ceph-osd-19.0.0-2879.g324baf8d.el9.x8 8.2 MB/s | 17 MB 00:02 2024-04-12T19:47:26.027 INFO:teuthology.orchestra.run.smithi169.stdout:(13/174): cephfs-shell-19.0.0-2879.g324baf8d.el 98 kB/s | 27 kB 00:00 2024-04-12T19:47:26.069 INFO:teuthology.orchestra.run.smithi169.stdout:(14/174): libcephfs-devel-19.0.0-2879.g324baf8d 483 kB/s | 32 kB 00:00 2024-04-12T19:47:26.137 INFO:teuthology.orchestra.run.smithi169.stdout:(15/174): libcephsqlite-19.0.0-2879.g324baf8d.e 2.4 MB/s | 164 kB 00:00 2024-04-12T19:47:26.220 INFO:teuthology.orchestra.run.smithi169.stdout:(16/174): libcephfs2-19.0.0-2879.g324baf8d.el9. 3.8 MB/s | 746 kB 00:00 2024-04-12T19:47:26.254 INFO:teuthology.orchestra.run.smithi169.stdout:(17/174): librados-devel-19.0.0-2879.g324baf8d. 1.1 MB/s | 127 kB 00:00 2024-04-12T19:47:26.304 INFO:teuthology.orchestra.run.smithi169.stdout:(18/174): libradosstriper1-19.0.0-2879.g324baf8 5.9 MB/s | 500 kB 00:00 2024-04-12T19:47:26.421 INFO:teuthology.orchestra.run.smithi169.stdout:(19/174): python3-ceph-argparse-19.0.0-2879.g32 390 kB/s | 45 kB 00:00 2024-04-12T19:47:26.513 INFO:teuthology.orchestra.run.smithi169.stdout:(20/174): python3-ceph-common-19.0.0-2879.g324b 1.5 MB/s | 135 kB 00:00 2024-04-12T19:47:26.605 INFO:teuthology.orchestra.run.smithi169.stdout:(21/174): python3-cephfs-19.0.0-2879.g324baf8d. 1.7 MB/s | 161 kB 00:00 2024-04-12T19:47:26.755 INFO:teuthology.orchestra.run.smithi169.stdout:(22/174): librgw2-19.0.0-2879.g324baf8d.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-12T19:47:26.797 INFO:teuthology.orchestra.run.smithi169.stdout:(23/174): python3-rados-19.0.0-2879.g324baf8d.e 1.6 MB/s | 320 kB 00:00 2024-04-12T19:47:26.864 INFO:teuthology.orchestra.run.smithi169.stdout:(24/174): python3-rbd-19.0.0-2879.g324baf8d.el9 2.7 MB/s | 297 kB 00:00 2024-04-12T19:47:26.898 INFO:teuthology.orchestra.run.smithi169.stdout:(25/174): python3-rgw-19.0.0-2879.g324baf8d.el9 996 kB/s | 99 kB 00:00 2024-04-12T19:47:26.939 INFO:teuthology.orchestra.run.smithi169.stdout:(26/174): rbd-fuse-19.0.0-2879.g324baf8d.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-12T19:47:27.031 INFO:teuthology.orchestra.run.smithi169.stdout:(27/174): rbd-nbd-19.0.0-2879.g324baf8d.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-12T19:47:27.107 INFO:teuthology.orchestra.run.smithi169.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2879.g 462 kB/s | 34 kB 00:00 2024-04-12T19:47:27.190 INFO:teuthology.orchestra.run.smithi169.stdout:(29/174): rbd-mirror-19.0.0-2879.g324baf8d.el9. 11 MB/s | 3.1 MB 00:00 2024-04-12T19:47:27.224 INFO:teuthology.orchestra.run.smithi169.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2879.g324baf8 1.2 MB/s | 142 kB 00:00 2024-04-12T19:47:27.608 INFO:teuthology.orchestra.run.smithi169.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2879.g324ba 8.5 MB/s | 3.5 MB 00:00 2024-04-12T19:47:27.717 INFO:teuthology.orchestra.run.smithi169.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2879.g32 2.2 MB/s | 249 kB 00:00 2024-04-12T19:47:27.808 INFO:teuthology.orchestra.run.smithi169.stdout:(33/174): ceph-mgr-rook-19.0.0-2879.g324baf8d.e 550 kB/s | 49 kB 00:00 2024-04-12T19:47:27.950 INFO:teuthology.orchestra.run.smithi169.stdout:(34/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-12T19:47:27.984 INFO:teuthology.orchestra.run.smithi169.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2879.g3 94 kB/s | 16 kB 00:00 2024-04-12T19:47:28.043 INFO:teuthology.orchestra.run.smithi169.stdout:(36/174): ceph-volume-19.0.0-2879.g324baf8d.el9 2.8 MB/s | 264 kB 00:00 2024-04-12T19:47:28.101 INFO:teuthology.orchestra.run.smithi169.stdout:(37/174): cephadm-19.0.0-2879.g324baf8d.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-12T19:47:28.126 INFO:teuthology.orchestra.run.smithi169.stdout:(38/174): cephfs-top-19.0.0-2879.g324baf8d.el9. 221 kB/s | 18 kB 00:00 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:Modular dependency problems: 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 1: conflicting requests 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 2: conflicting requests 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 3: conflicting requests 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 4: conflicting requests 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.198 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 5: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 6: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 7: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 8: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 9: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 10: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 11: conflicting requests 2024-04-12T19:47:28.199 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 12: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 13: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 14: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 15: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 16: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 17: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 18: conflicting requests 2024-04-12T19:47:28.200 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 19: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 20: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 21: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 22: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 23: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 24: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 25: conflicting requests 2024-04-12T19:47:28.201 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 26: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 27: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 28: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 29: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 30: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 31: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 32: conflicting requests 2024-04-12T19:47:28.202 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 33: conflicting requests 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 34: conflicting requests 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 35: conflicting requests 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 36: conflicting requests 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 37: conflicting requests 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.203 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 39: conflicting requests 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.204 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T19:47:28.205 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.206 INFO:teuthology.orchestra.run.smithi032.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T19:47:28.260 INFO:teuthology.orchestra.run.smithi169.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 111 kB/s | 18 kB 00:00 2024-04-12T19:47:28.310 INFO:teuthology.orchestra.run.smithi169.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 440 kB/s | 80 kB 00:00 2024-04-12T19:47:28.486 INFO:teuthology.orchestra.run.smithi169.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 834 kB/s | 187 kB 00:00 2024-04-12T19:47:28.553 INFO:teuthology.orchestra.run.smithi169.stdout:(42/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-12T19:47:28.614 INFO:teuthology.orchestra.run.smithi169.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-12T19:47:28.670 INFO:teuthology.orchestra.run.smithi169.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 218 kB 00:00 2024-04-12T19:47:28.729 INFO:teuthology.orchestra.run.smithi169.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 253 kB 00:00 2024-04-12T19:47:28.787 INFO:teuthology.orchestra.run.smithi169.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-12T19:47:28.812 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-12T19:47:28.814 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-12T19:47:28.836 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-12T19:47:28.837 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-12T19:47:28.838 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-12T19:47:28.840 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-12T19:47:28.842 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-12T19:47:28.846 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-12T19:47:28.847 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-12T19:47:28.913 INFO:teuthology.orchestra.run.smithi169.stdout:(47/174): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-04-12T19:47:28.957 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-04-12T19:47:28.971 INFO:teuthology.orchestra.run.smithi169.stdout:(48/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-12T19:47:28.974 INFO:teuthology.orchestra.run.smithi032.stdout:========================================================================================= 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout:========================================================================================= 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 6.6 k 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 5.4 M 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 889 k 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 145 k 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 1.5 M 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 142 k 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 3.5 M 2024-04-12T19:47:28.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 7.4 M 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 49 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 10 M 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 49 M 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 264 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 762 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 224 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 27 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 18 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 32 k 2024-04-12T19:47:28.976 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 746 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 127 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 161 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 320 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 297 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 99 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 85 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.1 M 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 171 k 2024-04-12T19:47:28.977 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-12T19:47:28.978 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.4 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 3.1 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-04-12T19:47:28.979 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 21 M 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 34 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 2.4 M 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 249 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 4.7 M 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 17 M 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2879.g324baf8d.el9 ceph-noarch 16 k 2024-04-12T19:47:28.980 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 25 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 164 k 2024-04-12T19:47:28.981 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 500 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 5.2 M 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-12T19:47:28.982 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-12T19:47:28.983 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 45 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:19.0.0-2879.g324baf8d.el9 ceph 135 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-12T19:47:28.984 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-12T19:47:28.985 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-12T19:47:28.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-12T19:47:28.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout:========================================================================================= 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout:Install 150 Packages 2024-04-12T19:47:28.988 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 24 Packages 2024-04-12T19:47:28.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:47:28.989 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 229 M 2024-04-12T19:47:28.989 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-04-12T19:47:28.996 INFO:teuthology.orchestra.run.smithi169.stdout:(49/174): python3-pycparser-2.20-6.el9.noarch.r 650 kB/s | 135 kB 00:00 2024-04-12T19:47:29.080 INFO:teuthology.orchestra.run.smithi169.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-12T19:47:29.372 INFO:teuthology.orchestra.run.smithi169.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-12T19:47:29.956 INFO:teuthology.orchestra.run.smithi169.stdout:(52/174): ceph-test-19.0.0-2879.g324baf8d.el9.x 11 MB/s | 49 MB 00:04 2024-04-12T19:47:29.989 INFO:teuthology.orchestra.run.smithi169.stdout:(53/174): boost-program-options-1.75.0-8.el9.x8 173 kB/s | 107 kB 00:00 2024-04-12T19:47:30.031 INFO:teuthology.orchestra.run.smithi169.stdout:(54/174): bison-3.7.4-5.el9.x86_64.rpm 993 kB/s | 943 kB 00:00 2024-04-12T19:47:30.157 INFO:teuthology.orchestra.run.smithi169.stdout:(55/174): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 33 kB 00:00 2024-04-12T19:47:30.198 INFO:teuthology.orchestra.run.smithi169.stdout:(56/174): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-04-12T19:47:30.200 INFO:teuthology.orchestra.run.smithi032.stdout:(1/174): ceph-19.0.0-2879.g324baf8d.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-12T19:47:30.215 INFO:teuthology.orchestra.run.smithi169.stdout:(57/174): elfutils-libelf-devel-0.190-2.el9.x86 83 kB/s | 21 kB 00:00 2024-04-12T19:47:30.266 INFO:teuthology.orchestra.run.smithi169.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-12T19:47:30.435 INFO:teuthology.orchestra.run.smithi032.stdout:(2/174): ceph-fuse-19.0.0-2879.g324baf8d.el9.x8 3.7 MB/s | 889 kB 00:00 2024-04-12T19:47:30.458 INFO:teuthology.orchestra.run.smithi169.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 237 kB/s | 45 kB 00:00 2024-04-12T19:47:30.525 INFO:teuthology.orchestra.run.smithi169.stdout:(60/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 519 kB/s | 160 kB 00:00 2024-04-12T19:47:30.526 INFO:teuthology.orchestra.run.smithi032.stdout:(3/174): ceph-immutable-object-cache-19.0.0-287 1.6 MB/s | 145 kB 00:00 2024-04-12T19:47:30.600 INFO:teuthology.orchestra.run.smithi169.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.8 MB/s | 3.0 MB 00:00 2024-04-12T19:47:30.650 INFO:teuthology.orchestra.run.smithi169.stdout:(62/174): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-12T19:47:30.677 INFO:teuthology.orchestra.run.smithi169.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-12T19:47:30.709 INFO:teuthology.orchestra.run.smithi169.stdout:(64/174): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-04-12T19:47:30.994 INFO:teuthology.orchestra.run.smithi032.stdout:(4/174): ceph-mds-19.0.0-2879.g324baf8d.el9.x86 5.1 MB/s | 2.4 MB 00:00 2024-04-12T19:47:31.134 INFO:teuthology.orchestra.run.smithi169.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 639 kB/s | 292 kB 00:00 2024-04-12T19:47:31.168 INFO:teuthology.orchestra.run.smithi169.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 411 kB/s | 188 kB 00:00 2024-04-12T19:47:31.193 INFO:teuthology.orchestra.run.smithi169.stdout:(67/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 87 kB/s | 47 kB 00:00 2024-04-12T19:47:31.219 INFO:teuthology.orchestra.run.smithi169.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 423 kB/s | 35 kB 00:00 2024-04-12T19:47:31.269 INFO:teuthology.orchestra.run.smithi032.stdout:(5/174): ceph-base-19.0.0-2879.g324baf8d.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-04-12T19:47:31.361 INFO:teuthology.orchestra.run.smithi032.stdout:(6/174): ceph-mgr-19.0.0-2879.g324baf8d.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-04-12T19:47:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:(69/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2024-04-12T19:47:31.561 INFO:teuthology.orchestra.run.smithi169.stdout:(70/174): postgresql-13.14-1.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-04-12T19:47:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:(71/174): openblas-openmp-0.3.21-2.el9.x86_64.r 5.8 MB/s | 4.7 MB 00:00 2024-04-12T19:47:32.185 INFO:teuthology.orchestra.run.smithi032.stdout:(7/174): ceph-common-19.0.0-2879.g324baf8d.el9. 9.2 MB/s | 21 MB 00:02 2024-04-12T19:47:32.185 INFO:teuthology.orchestra.run.smithi032.stdout:(8/174): ceph-mon-19.0.0-2879.g324baf8d.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-04-12T19:47:32.185 INFO:teuthology.orchestra.run.smithi169.stdout:(72/174): postgresql-private-libs-13.14-1.el9.x 296 kB/s | 133 kB 00:00 2024-04-12T19:47:32.185 INFO:teuthology.orchestra.run.smithi169.stdout:(73/174): postgresql-contrib-13.14-1.el9.x86_64 1.6 MB/s | 911 kB 00:00 2024-04-12T19:47:32.195 INFO:teuthology.orchestra.run.smithi169.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 700 kB/s | 87 kB 00:00 2024-04-12T19:47:32.246 INFO:teuthology.orchestra.run.smithi169.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-12T19:47:32.262 INFO:teuthology.orchestra.run.smithi032.stdout:(9/174): ceph-selinux-19.0.0-2879.g324baf8d.el9 306 kB/s | 25 kB 00:00 2024-04-12T19:47:32.321 INFO:teuthology.orchestra.run.smithi169.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-04-12T19:47:32.496 INFO:teuthology.orchestra.run.smithi169.stdout:(77/174): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-12T19:47:32.603 INFO:teuthology.orchestra.run.smithi169.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 244 kB/s | 26 kB 00:00 2024-04-12T19:47:32.639 INFO:teuthology.orchestra.run.smithi169.stdout:(79/174): python3-jinja2-2.11.3-5.el9.noarch.rp 784 kB/s | 248 kB 00:00 2024-04-12T19:47:32.664 INFO:teuthology.orchestra.run.smithi169.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 318 kB/s | 19 kB 00:00 2024-04-12T19:47:32.781 INFO:teuthology.orchestra.run.smithi169.stdout:(81/174): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-12T19:47:32.793 INFO:teuthology.orchestra.run.smithi032.stdout:(10/174): ceph-radosgw-19.0.0-2879.g324baf8d.el 13 MB/s | 10 MB 00:00 2024-04-12T19:47:32.806 INFO:teuthology.orchestra.run.smithi169.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 246 kB/s | 35 kB 00:00 2024-04-12T19:47:32.840 INFO:teuthology.orchestra.run.smithi169.stdout:(83/174): python3-mako-1.1.4-6.el9.noarch.rpm 861 kB/s | 172 kB 00:00 2024-04-12T19:47:32.846 INFO:teuthology.orchestra.run.smithi032.stdout:(11/174): cephfs-mirror-19.0.0-2879.g324baf8d.e 4.3 MB/s | 224 kB 00:00 2024-04-12T19:47:32.873 INFO:teuthology.orchestra.run.smithi169.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-12T19:47:32.907 INFO:teuthology.orchestra.run.smithi169.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-12T19:47:32.913 INFO:teuthology.orchestra.run.smithi032.stdout:(12/174): cephfs-shell-19.0.0-2879.g324baf8d.el 410 kB/s | 27 kB 00:00 2024-04-12T19:47:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:(13/174): libcephfs-devel-19.0.0-2879.g324baf8d 483 kB/s | 32 kB 00:00 2024-04-12T19:47:33.080 INFO:teuthology.orchestra.run.smithi032.stdout:(14/174): libcephfs2-19.0.0-2879.g324baf8d.el9. 7.4 MB/s | 746 kB 00:00 2024-04-12T19:47:33.099 INFO:teuthology.orchestra.run.smithi169.stdout:(86/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-12T19:47:33.147 INFO:teuthology.orchestra.run.smithi032.stdout:(15/174): libcephsqlite-19.0.0-2879.g324baf8d.e 2.4 MB/s | 164 kB 00:00 2024-04-12T19:47:33.222 INFO:teuthology.orchestra.run.smithi032.stdout:(16/174): librados-devel-19.0.0-2879.g324baf8d. 1.7 MB/s | 127 kB 00:00 2024-04-12T19:47:33.306 INFO:teuthology.orchestra.run.smithi032.stdout:(17/174): libradosstriper1-19.0.0-2879.g324baf8 5.9 MB/s | 500 kB 00:00 2024-04-12T19:47:33.358 INFO:teuthology.orchestra.run.smithi169.stdout:(87/174): python3-packaging-20.9-5.el9.noarch.r 160 kB/s | 77 kB 00:00 2024-04-12T19:47:33.391 INFO:teuthology.orchestra.run.smithi169.stdout:(88/174): python3-prettytable-0.7.2-27.el9.noar 87 kB/s | 42 kB 00:00 2024-04-12T19:47:33.417 INFO:teuthology.orchestra.run.smithi169.stdout:(89/174): python3-protobuf-3.14.0-13.el9.noarch 850 kB/s | 269 kB 00:00 2024-04-12T19:47:33.450 INFO:teuthology.orchestra.run.smithi169.stdout:(90/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-12T19:47:33.509 INFO:teuthology.orchestra.run.smithi169.stdout:(91/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-12T19:47:33.526 INFO:teuthology.orchestra.run.smithi169.stdout:(92/174): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-04-12T19:47:33.543 INFO:teuthology.orchestra.run.smithi169.stdout:(93/174): python3-requests-oauthlib-1.3.0-12.el 586 kB/s | 54 kB 00:00 2024-04-12T19:47:33.585 INFO:teuthology.orchestra.run.smithi169.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 719 kB/s | 42 kB 00:00 2024-04-12T19:47:33.739 INFO:teuthology.orchestra.run.smithi032.stdout:(18/174): librgw2-19.0.0-2879.g324baf8d.el9.x86 12 MB/s | 5.2 MB 00:00 2024-04-12T19:47:33.966 INFO:teuthology.orchestra.run.smithi032.stdout:(19/174): ceph-osd-19.0.0-2879.g324baf8d.el9.x8 6.5 MB/s | 17 MB 00:02 2024-04-12T19:47:33.990 INFO:teuthology.orchestra.run.smithi032.stdout:(20/174): python3-ceph-argparse-19.0.0-2879.g32 181 kB/s | 45 kB 00:00 2024-04-12T19:47:34.023 INFO:teuthology.orchestra.run.smithi032.stdout:(21/174): python3-ceph-common-19.0.0-2879.g324b 2.3 MB/s | 135 kB 00:00 2024-04-12T19:47:34.057 INFO:teuthology.orchestra.run.smithi032.stdout:(22/174): python3-cephfs-19.0.0-2879.g324baf8d. 2.4 MB/s | 161 kB 00:00 2024-04-12T19:47:34.141 INFO:teuthology.orchestra.run.smithi032.stdout:(23/174): python3-rados-19.0.0-2879.g324baf8d.e 2.7 MB/s | 320 kB 00:00 2024-04-12T19:47:34.175 INFO:teuthology.orchestra.run.smithi032.stdout:(24/174): python3-rbd-19.0.0-2879.g324baf8d.el9 2.5 MB/s | 297 kB 00:00 2024-04-12T19:47:34.193 INFO:teuthology.orchestra.run.smithi169.stdout:(95/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-12T19:47:34.200 INFO:teuthology.orchestra.run.smithi032.stdout:(25/174): python3-rgw-19.0.0-2879.g324baf8d.el9 1.7 MB/s | 99 kB 00:00 2024-04-12T19:47:34.233 INFO:teuthology.orchestra.run.smithi032.stdout:(26/174): rbd-fuse-19.0.0-2879.g324baf8d.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-12T19:47:34.235 INFO:teuthology.orchestra.run.smithi169.stdout:(96/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 64 kB/s | 44 kB 00:00 2024-04-12T19:47:34.302 INFO:teuthology.orchestra.run.smithi169.stdout:(97/174): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-04-12T19:47:34.309 INFO:teuthology.orchestra.run.smithi032.stdout:(27/174): rbd-nbd-19.0.0-2879.g324baf8d.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-12T19:47:34.327 INFO:teuthology.orchestra.run.smithi169.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 519 kB/s | 47 kB 00:00 2024-04-12T19:47:34.359 INFO:teuthology.orchestra.run.smithi032.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2879.g 694 kB/s | 34 kB 00:00 2024-04-12T19:47:34.386 INFO:teuthology.orchestra.run.smithi169.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 276 kB/s | 221 kB 00:00 2024-04-12T19:47:34.426 INFO:teuthology.orchestra.run.smithi032.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2879.g324baf8 2.1 MB/s | 142 kB 00:00 2024-04-12T19:47:34.445 INFO:teuthology.orchestra.run.smithi169.stdout:(100/174): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-04-12T19:47:34.470 INFO:teuthology.orchestra.run.smithi169.stdout:(101/174): uuid-1.6.2-55.el9.x86_64.rpm 408 kB/s | 58 kB 00:00 2024-04-12T19:47:34.495 INFO:teuthology.orchestra.run.smithi169.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-04-12T19:47:34.579 INFO:teuthology.orchestra.run.smithi169.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 270 kB/s | 22 kB 00:00 2024-04-12T19:47:34.593 INFO:teuthology.orchestra.run.smithi032.stdout:(30/174): rbd-mirror-19.0.0-2879.g324baf8d.el9. 7.9 MB/s | 3.1 MB 00:00 2024-04-12T19:47:34.693 INFO:teuthology.orchestra.run.smithi032.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2879.g324ba 13 MB/s | 3.5 MB 00:00 2024-04-12T19:47:34.696 INFO:teuthology.orchestra.run.smithi169.stdout:(104/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 201 kB/s | 45 kB 00:00 2024-04-12T19:47:34.721 INFO:teuthology.orchestra.run.smithi169.stdout:(105/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 231 kB/s | 64 kB 00:00 2024-04-12T19:47:34.760 INFO:teuthology.orchestra.run.smithi032.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2879.g32 3.7 MB/s | 249 kB 00:00 2024-04-12T19:47:34.763 INFO:teuthology.orchestra.run.smithi169.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-12T19:47:34.819 INFO:teuthology.orchestra.run.smithi032.stdout:(33/174): ceph-mgr-rook-19.0.0-2879.g324baf8d.e 861 kB/s | 49 kB 00:00 2024-04-12T19:47:34.877 INFO:teuthology.orchestra.run.smithi032.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2879.g3 283 kB/s | 16 kB 00:00 2024-04-12T19:47:34.944 INFO:teuthology.orchestra.run.smithi032.stdout:(35/174): ceph-volume-19.0.0-2879.g324baf8d.el9 3.9 MB/s | 264 kB 00:00 2024-04-12T19:47:35.037 INFO:teuthology.orchestra.run.smithi032.stdout:(36/174): cephadm-19.0.0-2879.g324baf8d.el9.noa 8.2 MB/s | 762 kB 00:00 2024-04-12T19:47:35.087 INFO:teuthology.orchestra.run.smithi032.stdout:(37/174): cephfs-top-19.0.0-2879.g324baf8d.el9. 371 kB/s | 18 kB 00:00 2024-04-12T19:47:35.180 INFO:teuthology.orchestra.run.smithi169.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 47 kB/s | 19 kB 00:00 2024-04-12T19:47:35.221 INFO:teuthology.orchestra.run.smithi032.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 133 kB/s | 18 kB 00:00 2024-04-12T19:47:35.405 INFO:teuthology.orchestra.run.smithi169.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 452 kB/s | 309 kB 00:00 2024-04-12T19:47:35.421 INFO:teuthology.orchestra.run.smithi032.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-12T19:47:35.439 INFO:teuthology.orchestra.run.smithi169.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 739 kB/s | 548 kB 00:00 2024-04-12T19:47:35.506 INFO:teuthology.orchestra.run.smithi169.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-12T19:47:35.539 INFO:teuthology.orchestra.run.smithi169.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-04-12T19:47:35.614 INFO:teuthology.orchestra.run.smithi169.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-04-12T19:47:35.656 INFO:teuthology.orchestra.run.smithi169.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-12T19:47:35.763 INFO:teuthology.orchestra.run.smithi032.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 148 kB/s | 80 kB 00:00 2024-04-12T19:47:35.798 INFO:teuthology.orchestra.run.smithi032.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 501 kB/s | 187 kB 00:00 2024-04-12T19:47:35.823 INFO:teuthology.orchestra.run.smithi169.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-12T19:47:35.866 INFO:teuthology.orchestra.run.smithi169.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-04-12T19:47:35.966 INFO:teuthology.orchestra.run.smithi169.stdout:(116/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-12T19:47:35.991 INFO:teuthology.orchestra.run.smithi169.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 261 kB/s | 43 kB 00:00 2024-04-12T19:47:36.016 INFO:teuthology.orchestra.run.smithi169.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 215 kB/s | 32 kB 00:00 2024-04-12T19:47:36.067 INFO:teuthology.orchestra.run.smithi169.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-12T19:47:36.165 INFO:teuthology.orchestra.run.smithi169.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-12T19:47:36.180 INFO:teuthology.orchestra.run.smithi032.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 511 kB/s | 195 kB 00:00 2024-04-12T19:47:36.200 INFO:teuthology.orchestra.run.smithi169.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-12T19:47:36.234 INFO:teuthology.orchestra.run.smithi169.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-12T19:47:36.239 INFO:teuthology.orchestra.run.smithi032.stdout:(43/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-12T19:47:36.259 INFO:teuthology.orchestra.run.smithi169.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 379 kB/s | 35 kB 00:00 2024-04-12T19:47:36.318 INFO:teuthology.orchestra.run.smithi169.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-12T19:47:36.369 INFO:teuthology.orchestra.run.smithi169.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-12T19:47:36.469 INFO:teuthology.orchestra.run.smithi169.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-04-12T19:47:36.561 INFO:teuthology.orchestra.run.smithi169.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 44 kB/s | 11 kB 00:00 2024-04-12T19:47:36.614 INFO:teuthology.orchestra.run.smithi032.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 505 kB/s | 218 kB 00:00 2024-04-12T19:47:36.617 INFO:teuthology.orchestra.run.smithi169.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 71 kB/s | 18 kB 00:00 2024-04-12T19:47:36.644 INFO:teuthology.orchestra.run.smithi169.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 133 kB/s | 23 kB 00:00 2024-04-12T19:47:36.670 INFO:teuthology.orchestra.run.smithi169.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-04-12T19:47:36.720 INFO:teuthology.orchestra.run.smithi169.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 193 kB/s | 20 kB 00:00 2024-04-12T19:47:36.779 INFO:teuthology.orchestra.run.smithi169.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-04-12T19:47:36.804 INFO:teuthology.orchestra.run.smithi169.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-04-12T19:47:36.829 INFO:teuthology.orchestra.run.smithi169.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 731 kB/s | 79 kB 00:00 2024-04-12T19:47:36.896 INFO:teuthology.orchestra.run.smithi169.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-04-12T19:47:36.931 INFO:teuthology.orchestra.run.smithi169.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-12T19:47:36.956 INFO:teuthology.orchestra.run.smithi169.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-04-12T19:47:36.992 INFO:teuthology.orchestra.run.smithi169.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-04-12T19:47:37.062 INFO:teuthology.orchestra.run.smithi169.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 215 kB/s | 28 kB 00:00 2024-04-12T19:47:37.098 INFO:teuthology.orchestra.run.smithi169.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-04-12T19:47:37.132 INFO:teuthology.orchestra.run.smithi169.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-12T19:47:37.165 INFO:teuthology.orchestra.run.smithi169.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 581 kB/s | 59 kB 00:00 2024-04-12T19:47:37.190 INFO:teuthology.orchestra.run.smithi032.stdout:(45/174): ceph-test-19.0.0-2879.g324baf8d.el9.x 9.9 MB/s | 49 MB 00:04 2024-04-12T19:47:37.198 INFO:teuthology.orchestra.run.smithi169.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-12T19:47:37.241 INFO:teuthology.orchestra.run.smithi169.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-12T19:47:37.291 INFO:teuthology.orchestra.run.smithi169.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-12T19:47:37.307 INFO:teuthology.orchestra.run.smithi032.stdout:(46/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 254 kB/s | 253 kB 00:00 2024-04-12T19:47:37.316 INFO:teuthology.orchestra.run.smithi169.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-04-12T19:47:37.358 INFO:teuthology.orchestra.run.smithi169.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-12T19:47:37.387 INFO:teuthology.orchestra.run.smithi169.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 210 kB/s | 20 kB 00:00 2024-04-12T19:47:37.429 INFO:teuthology.orchestra.run.smithi169.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-12T19:47:37.521 INFO:teuthology.orchestra.run.smithi169.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.8 MB/s | 1.6 MB 00:00 2024-04-12T19:47:37.682 INFO:teuthology.orchestra.run.smithi032.stdout:(47/174): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-04-12T19:47:37.724 INFO:teuthology.orchestra.run.smithi032.stdout:(48/174): python3-ply-3.11-14.el9.noarch.rpm 230 kB/s | 106 kB 00:00 2024-04-12T19:47:37.824 INFO:teuthology.orchestra.run.smithi032.stdout:(49/174): python3-pycparser-2.20-6.el9.noarch.r 296 kB/s | 135 kB 00:00 2024-04-12T19:47:37.879 INFO:teuthology.orchestra.run.smithi169.stdout:(151/174): librados2-19.0.0-2879.g324baf8d.el9. 6.9 MB/s | 3.4 MB 00:00 2024-04-12T19:47:38.041 INFO:teuthology.orchestra.run.smithi032.stdout:(50/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-12T19:47:38.064 INFO:teuthology.orchestra.run.smithi169.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-04-12T19:47:38.074 INFO:teuthology.orchestra.run.smithi032.stdout:(51/174): python3-requests-2.25.1-8.el9.noarch. 319 kB/s | 125 kB 00:00 2024-04-12T19:47:38.133 INFO:teuthology.orchestra.run.smithi032.stdout:(52/174): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-12T19:47:38.167 INFO:teuthology.orchestra.run.smithi032.stdout:(53/174): python3-urllib3-1.26.5-5.el9.noarch.r 577 kB/s | 215 kB 00:00 2024-04-12T19:47:38.189 INFO:teuthology.orchestra.run.smithi169.stdout:(153/174): librbd1-19.0.0-2879.g324baf8d.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-12T19:47:38.192 INFO:teuthology.orchestra.run.smithi032.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 372 kB/s | 21 kB 00:00 2024-04-12T19:47:38.214 INFO:teuthology.orchestra.run.smithi169.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 281 kB/s | 42 kB 00:00 2024-04-12T19:47:38.217 INFO:teuthology.orchestra.run.smithi032.stdout:(55/174): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-04-12T19:47:38.242 INFO:teuthology.orchestra.run.smithi032.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-04-12T19:47:38.273 INFO:teuthology.orchestra.run.smithi169.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-04-12T19:47:38.284 INFO:teuthology.orchestra.run.smithi032.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 448 kB/s | 18 kB 00:00 2024-04-12T19:47:38.318 INFO:teuthology.orchestra.run.smithi032.stdout:(58/174): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-12T19:47:38.343 INFO:teuthology.orchestra.run.smithi032.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-12T19:47:38.369 INFO:teuthology.orchestra.run.smithi032.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 918 kB/s | 45 kB 00:00 2024-04-12T19:47:38.378 INFO:teuthology.orchestra.run.smithi169.stdout:(156/174): NetworkManager-libnm-1.47.4-1.el9.x8 3.7 MB/s | 1.8 MB 00:00 2024-04-12T19:47:38.407 INFO:teuthology.orchestra.run.smithi169.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 274 kB/s | 36 kB 00:00 2024-04-12T19:47:38.419 INFO:teuthology.orchestra.run.smithi032.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-12T19:47:38.440 INFO:teuthology.orchestra.run.smithi169.stdout:(158/174): NetworkManager-tui-1.47.4-1.el9.x86_ 996 kB/s | 249 kB 00:00 2024-04-12T19:47:38.469 INFO:teuthology.orchestra.run.smithi032.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-12T19:47:38.473 INFO:teuthology.orchestra.run.smithi169.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-04-12T19:47:38.495 INFO:teuthology.orchestra.run.smithi032.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 341 kB/s | 25 kB 00:00 2024-04-12T19:47:38.499 INFO:teuthology.orchestra.run.smithi169.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 397 kB/s | 23 kB 00:00 2024-04-12T19:47:38.520 INFO:teuthology.orchestra.run.smithi032.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 960 kB/s | 47 kB 00:00 2024-04-12T19:47:38.532 INFO:teuthology.orchestra.run.smithi169.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-04-12T19:47:38.558 INFO:teuthology.orchestra.run.smithi169.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-12T19:47:38.574 INFO:teuthology.orchestra.run.smithi032.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.6 MB/s | 292 kB 00:00 2024-04-12T19:47:38.583 INFO:teuthology.orchestra.run.smithi169.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-04-12T19:47:38.608 INFO:teuthology.orchestra.run.smithi169.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-04-12T19:47:38.612 INFO:teuthology.orchestra.run.smithi032.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-12T19:47:38.634 INFO:teuthology.orchestra.run.smithi169.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 853 kB/s | 64 kB 00:00 2024-04-12T19:47:38.637 INFO:teuthology.orchestra.run.smithi032.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 567 kB/s | 35 kB 00:00 2024-04-12T19:47:38.659 INFO:teuthology.orchestra.run.smithi169.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 383 kB/s | 29 kB 00:00 2024-04-12T19:47:38.738 INFO:teuthology.orchestra.run.smithi032.stdout:(68/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-04-12T19:47:38.768 INFO:teuthology.orchestra.run.smithi169.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-04-12T19:47:38.846 INFO:teuthology.orchestra.run.smithi032.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-12T19:47:38.901 INFO:teuthology.orchestra.run.smithi169.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 2.2 MB 00:00 2024-04-12T19:47:38.922 INFO:teuthology.orchestra.run.smithi032.stdout:(70/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-12T19:47:38.934 INFO:teuthology.orchestra.run.smithi169.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-04-12T19:47:38.960 INFO:teuthology.orchestra.run.smithi169.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 539 kB/s | 31 kB 00:00 2024-04-12T19:47:39.002 INFO:teuthology.orchestra.run.smithi169.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2024-04-12T19:47:39.005 INFO:teuthology.orchestra.run.smithi032.stdout:(71/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2024-04-12T19:47:39.094 INFO:teuthology.orchestra.run.smithi169.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2024-04-12T19:47:39.097 INFO:teuthology.orchestra.run.smithi032.stdout:(72/174): openblas-openmp-0.3.21-2.el9.x86_64.r 9.8 MB/s | 4.7 MB 00:00 2024-04-12T19:47:39.114 INFO:teuthology.orchestra.run.smithi032.stdout:(73/174): postgresql-private-libs-13.14-1.el9.x 696 kB/s | 133 kB 00:00 2024-04-12T19:47:39.156 INFO:teuthology.orchestra.run.smithi032.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-12T19:47:39.203 INFO:teuthology.orchestra.run.smithi169.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.2 MB/s | 2.3 MB 00:00 2024-04-12T19:47:39.223 INFO:teuthology.orchestra.run.smithi032.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-12T19:47:39.228 INFO:teuthology.orchestra.run.smithi169.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 331 kB/s | 74 kB 00:00 2024-04-12T19:47:39.240 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-12T19:47:39.241 INFO:teuthology.orchestra.run.smithi169.stdout:Total 13 MB/s | 229 MB 00:17 2024-04-12T19:47:39.273 INFO:teuthology.orchestra.run.smithi032.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.9 MB/s | 245 kB 00:00 2024-04-12T19:47:39.324 INFO:teuthology.orchestra.run.smithi032.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-12T19:47:39.366 INFO:teuthology.orchestra.run.smithi032.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 630 kB/s | 26 kB 00:00 2024-04-12T19:47:39.407 INFO:teuthology.orchestra.run.smithi032.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 470 kB/s | 19 kB 00:00 2024-04-12T19:47:39.458 INFO:teuthology.orchestra.run.smithi032.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-12T19:47:39.575 INFO:teuthology.orchestra.run.smithi032.stdout:(81/174): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-12T19:47:39.600 INFO:teuthology.orchestra.run.smithi032.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 246 kB/s | 35 kB 00:00 2024-04-12T19:47:39.650 INFO:teuthology.orchestra.run.smithi032.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-12T19:47:39.759 INFO:teuthology.orchestra.run.smithi032.stdout:(84/174): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 6.0 MB 00:00 2024-04-12T19:47:39.932 INFO:teuthology.orchestra.run.smithi032.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 791 kB/s | 222 kB 00:00 2024-04-12T19:47:39.960 INFO:teuthology.orchestra.run.smithi032.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 387 kB/s | 77 kB 00:00 2024-04-12T19:47:39.976 INFO:teuthology.orchestra.run.smithi032.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 987 kB/s | 42 kB 00:00 2024-04-12T19:47:40.002 INFO:teuthology.orchestra.run.smithi032.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 6.4 MB/s | 269 kB 00:00 2024-04-12T19:47:40.027 INFO:teuthology.orchestra.run.smithi032.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-12T19:47:40.060 INFO:teuthology.orchestra.run.smithi032.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-12T19:47:40.095 INFO:teuthology.orchestra.run.smithi032.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 769 kB/s | 51 kB 00:00 2024-04-12T19:47:40.120 INFO:teuthology.orchestra.run.smithi032.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 925 kB/s | 54 kB 00:00 2024-04-12T19:47:40.162 INFO:teuthology.orchestra.run.smithi032.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-12T19:47:40.204 INFO:teuthology.orchestra.run.smithi032.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-12T19:47:40.254 INFO:teuthology.orchestra.run.smithi032.stdout:(95/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-04-12T19:47:40.363 INFO:teuthology.orchestra.run.smithi032.stdout:(96/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.5 MB/s | 5.1 MB 00:00 2024-04-12T19:47:40.388 INFO:teuthology.orchestra.run.smithi032.stdout:(97/174): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-04-12T19:47:40.413 INFO:teuthology.orchestra.run.smithi032.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-04-12T19:47:40.447 INFO:teuthology.orchestra.run.smithi032.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-12T19:47:40.472 INFO:teuthology.orchestra.run.smithi032.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-12T19:47:40.497 INFO:teuthology.orchestra.run.smithi032.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-04-12T19:47:40.523 INFO:teuthology.orchestra.run.smithi032.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-12T19:47:40.548 INFO:teuthology.orchestra.run.smithi032.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 913 kB/s | 45 kB 00:00 2024-04-12T19:47:40.936 INFO:teuthology.orchestra.run.smithi032.stdout:(104/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-12T19:47:40.961 INFO:teuthology.orchestra.run.smithi032.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2024-04-12T19:47:41.037 INFO:teuthology.orchestra.run.smithi032.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-12T19:47:41.304 INFO:teuthology.orchestra.run.smithi032.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-12T19:47:41.529 INFO:teuthology.orchestra.run.smithi032.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-04-12T19:47:41.572 INFO:teuthology.orchestra.run.smithi032.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 865 kB/s | 548 kB 00:00 2024-04-12T19:47:41.630 INFO:teuthology.orchestra.run.smithi032.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-12T19:47:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-04-12T19:47:41.731 INFO:teuthology.orchestra.run.smithi032.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2024-04-12T19:47:41.789 INFO:teuthology.orchestra.run.smithi032.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-12T19:47:41.957 INFO:teuthology.orchestra.run.smithi032.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-12T19:47:41.998 INFO:teuthology.orchestra.run.smithi032.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-12T19:47:42.090 INFO:teuthology.orchestra.run.smithi032.stdout:(116/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-12T19:47:42.116 INFO:teuthology.orchestra.run.smithi032.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2024-04-12T19:47:42.141 INFO:teuthology.orchestra.run.smithi032.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 227 kB/s | 32 kB 00:00 2024-04-12T19:47:42.233 INFO:teuthology.orchestra.run.smithi032.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-04-12T19:47:42.266 INFO:teuthology.orchestra.run.smithi032.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-12T19:47:42.300 INFO:teuthology.orchestra.run.smithi032.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-12T19:47:42.342 INFO:teuthology.orchestra.run.smithi032.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-12T19:47:42.392 INFO:teuthology.orchestra.run.smithi032.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-04-12T19:47:42.426 INFO:teuthology.orchestra.run.smithi032.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-04-12T19:47:42.509 INFO:teuthology.orchestra.run.smithi032.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-12T19:47:42.551 INFO:teuthology.orchestra.run.smithi032.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-04-12T19:47:42.577 INFO:teuthology.orchestra.run.smithi032.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 787 kB/s | 144 kB 00:00 2024-04-12T19:47:42.602 INFO:teuthology.orchestra.run.smithi032.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-12T19:47:42.677 INFO:teuthology.orchestra.run.smithi032.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-12T19:47:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-12T19:47:42.728 INFO:teuthology.orchestra.run.smithi032.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-04-12T19:47:42.828 INFO:teuthology.orchestra.run.smithi032.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-04-12T19:47:42.853 INFO:teuthology.orchestra.run.smithi032.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 373 kB/s | 46 kB 00:00 2024-04-12T19:47:42.879 INFO:teuthology.orchestra.run.smithi032.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 528 kB/s | 79 kB 00:00 2024-04-12T19:47:42.914 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-12T19:47:42.963 INFO:teuthology.orchestra.run.smithi032.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 434 kB/s | 58 kB 00:00 2024-04-12T19:47:42.987 INFO:teuthology.orchestra.run.smithi032.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-04-12T19:47:43.021 INFO:teuthology.orchestra.run.smithi032.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-12T19:47:43.063 INFO:teuthology.orchestra.run.smithi032.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 902 kB/s | 90 kB 00:00 2024-04-12T19:47:43.088 INFO:teuthology.orchestra.run.smithi032.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-12T19:47:43.122 INFO:teuthology.orchestra.run.smithi032.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-12T19:47:43.189 INFO:teuthology.orchestra.run.smithi032.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-12T19:47:43.214 INFO:teuthology.orchestra.run.smithi032.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-04-12T19:47:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-12T19:47:43.290 INFO:teuthology.orchestra.run.smithi032.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-12T19:47:43.307 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-12T19:47:43.307 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-12T19:47:43.323 INFO:teuthology.orchestra.run.smithi032.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-12T19:47:43.349 INFO:teuthology.orchestra.run.smithi032.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 827 kB/s | 90 kB 00:00 2024-04-12T19:47:43.407 INFO:teuthology.orchestra.run.smithi032.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-12T19:47:43.424 INFO:teuthology.orchestra.run.smithi032.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-12T19:47:43.458 INFO:teuthology.orchestra.run.smithi032.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-12T19:47:43.550 INFO:teuthology.orchestra.run.smithi032.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-12T19:47:43.850 INFO:teuthology.orchestra.run.smithi032.stdout:(151/174): librados2-19.0.0-2879.g324baf8d.el9. 8.0 MB/s | 3.4 MB 00:00 2024-04-12T19:47:44.009 INFO:teuthology.orchestra.run.smithi032.stdout:(152/174): librbd1-19.0.0-2879.g324baf8d.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-12T19:47:44.234 INFO:teuthology.orchestra.run.smithi032.stdout:(153/174): NetworkManager-1.47.4-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-04-12T19:47:44.293 INFO:teuthology.orchestra.run.smithi032.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 149 kB/s | 42 kB 00:00 2024-04-12T19:47:44.468 INFO:teuthology.orchestra.run.smithi032.stdout:(155/174): NetworkManager-libnm-1.47.4-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-04-12T19:47:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:(156/174): NetworkManager-tui-1.47.4-1.el9.x86_ 514 kB/s | 249 kB 00:00 2024-04-12T19:47:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 154 kB/s | 36 kB 00:00 2024-04-12T19:47:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 172 kB/s | 71 kB 00:00 2024-04-12T19:47:45.161 INFO:teuthology.orchestra.run.smithi032.stdout:(159/174): libacl-2.3.1-4.el9.x86_64.rpm 88 kB/s | 23 kB 00:00 2024-04-12T19:47:45.195 INFO:teuthology.orchestra.run.smithi032.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 405 kB/s | 192 kB 00:00 2024-04-12T19:47:45.346 INFO:teuthology.orchestra.run.smithi032.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 426 kB/s | 256 kB 00:00 2024-04-12T19:47:45.546 INFO:teuthology.orchestra.run.smithi032.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 283 kB/s | 108 kB 00:00 2024-04-12T19:47:45.688 INFO:teuthology.orchestra.run.smithi032.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 315 kB/s | 155 kB 00:00 2024-04-12T19:47:45.755 INFO:teuthology.orchestra.run.smithi032.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 332 kB/s | 136 kB 00:00 2024-04-12T19:47:45.825 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-12T19:47:45.825 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-12T19:47:45.872 INFO:teuthology.orchestra.run.smithi032.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 197 kB/s | 64 kB 00:00 2024-04-12T19:47:45.988 INFO:teuthology.orchestra.run.smithi032.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 96 kB/s | 29 kB 00:00 2024-04-12T19:47:46.347 INFO:teuthology.orchestra.run.smithi032.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 2.3 MB/s | 1.2 MB 00:00 2024-04-12T19:47:46.614 INFO:teuthology.orchestra.run.smithi032.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.1 MB/s | 2.3 MB 00:00 2024-04-12T19:47:46.706 INFO:teuthology.orchestra.run.smithi032.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.8 MB/s | 2.2 MB 00:00 2024-04-12T19:47:46.823 INFO:teuthology.orchestra.run.smithi032.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 981 kB/s | 465 kB 00:00 2024-04-12T19:47:46.923 INFO:teuthology.orchestra.run.smithi032.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 101 kB/s | 31 kB 00:00 2024-04-12T19:47:47.015 INFO:teuthology.orchestra.run.smithi032.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 813 kB/s | 74 kB 00:00 2024-04-12T19:47:47.182 INFO:teuthology.orchestra.run.smithi032.stdout:(173/174): xfsdump-3.1.12-4.el9.x86_64.rpm 795 kB/s | 323 kB 00:00 2024-04-12T19:47:47.399 INFO:teuthology.orchestra.run.smithi032.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.1 MB/s | 1.1 MB 00:00 2024-04-12T19:47:47.406 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-04-12T19:47:47.406 INFO:teuthology.orchestra.run.smithi032.stdout:Total 12 MB/s | 229 MB 00:18 2024-04-12T19:47:51.285 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-04-12T19:47:51.696 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-04-12T19:47:51.696 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-04-12T19:47:53.685 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-12T19:47:53.818 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-12T19:47:53.867 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-12T19:47:53.913 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-12T19:47:53.960 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-12T19:47:54.054 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-12T19:47:54.156 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-12T19:47:54.214 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-12T19:47:54.245 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-12T19:47:54.246 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-04-12T19:47:54.247 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-04-12T19:47:54.319 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-12T19:47:54.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-12T19:47:54.799 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:47:54.960 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:47:54.992 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:47:55.112 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:47:55.149 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 12/198 2024-04-12T19:47:55.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-12T19:47:55.516 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-12T19:47:55.572 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-12T19:47:55.597 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:47:55.744 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:47:55.781 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 17/198 2024-04-12T19:47:55.870 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 17/198 2024-04-12T19:47:55.917 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-12T19:47:55.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-12T19:47:56.016 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-12T19:47:56.339 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-12T19:47:56.394 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-12T19:47:56.442 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-12T19:47:56.714 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-12T19:47:56.923 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-12T19:47:57.011 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-12T19:47:57.204 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:47:57.259 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:47:57.603 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:47:57.803 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-12T19:47:57.954 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-12T19:47:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-12T19:47:58.048 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-12T19:47:58.153 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-12T19:47:58.194 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-12T19:47:58.256 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-12T19:47:58.298 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-12T19:47:58.330 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-12T19:47:58.464 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-12T19:47:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-12T19:47:58.574 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-12T19:47:58.634 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-12T19:47:58.742 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-12T19:47:58.778 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-2879.g324baf8d.el9. 39/198 2024-04-12T19:47:58.825 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el 40/198 2024-04-12T19:47:58.859 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_6 41/198 2024-04-12T19:47:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-12T19:47:59.028 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-12T19:47:59.125 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-12T19:47:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-12T19:47:59.374 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-12T19:48:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-12T19:48:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-12T19:48:00.306 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-12T19:48:00.373 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-12T19:48:00.402 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-12T19:48:00.457 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-12T19:48:00.525 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-12T19:48:00.582 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_6 53/198 2024-04-12T19:48:00.631 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-12T19:48:01.049 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-12T19:48:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-12T19:48:01.410 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-12T19:48:01.474 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-12T19:48:01.686 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-12T19:48:01.740 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-12T19:48:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-04-12T19:48:01.817 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-12T19:48:01.910 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-12T19:48:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-12T19:48:01.956 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-12T19:48:01.997 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-12T19:48:01.997 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-12T19:48:02.043 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-12T19:48:02.064 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-12T19:48:02.090 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-12T19:48:02.104 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-12T19:48:02.174 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-12T19:48:02.179 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-12T19:48:02.235 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-12T19:48:02.292 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-12T19:48:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-12T19:48:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-12T19:48:02.434 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-12T19:48:02.485 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-12T19:48:02.588 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-12T19:48:02.588 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-12T19:48:02.588 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-12T19:48:02.700 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-12T19:48:02.747 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-12T19:48:02.793 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-12T19:48:02.837 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-12T19:48:02.870 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-12T19:48:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-12T19:48:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-12T19:48:03.174 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:48:03.213 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-12T19:48:03.285 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-12T19:48:03.323 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-12T19:48:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:48:03.360 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-12T19:48:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:48:03.418 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-12T19:48:03.482 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-12T19:48:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:48:03.522 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-12T19:48:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 12/198 2024-04-12T19:48:03.568 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-12T19:48:03.620 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-12T19:48:03.630 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-12T19:48:03.696 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-12T19:48:03.751 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-12T19:48:03.898 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-12T19:48:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-12T19:48:03.955 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-12T19:48:03.996 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-12T19:48:03.998 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-12T19:48:04.021 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:48:04.060 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-12T19:48:04.183 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:48:04.214 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 17/198 2024-04-12T19:48:04.223 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-12T19:48:04.286 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-12T19:48:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 17/198 2024-04-12T19:48:04.375 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-12T19:48:04.407 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-12T19:48:04.471 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-12T19:48:04.735 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-12T19:48:04.791 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-12T19:48:04.842 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-12T19:48:05.169 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-12T19:48:05.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-12T19:48:05.319 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-12T19:48:05.353 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-12T19:48:05.392 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-12T19:48:05.482 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-12T19:48:05.483 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-12T19:48:05.549 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-12T19:48:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-12T19:48:05.674 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:48:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-12T19:48:05.745 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:48:05.937 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-12T19:48:05.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-12T19:48:06.070 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-12T19:48:06.071 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-12T19:48:06.266 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-12T19:48:06.414 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-12T19:48:06.451 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-12T19:48:06.500 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-12T19:48:06.605 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-12T19:48:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-12T19:48:06.735 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-12T19:48:06.786 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-12T19:48:06.825 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-12T19:48:06.880 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-12T19:48:06.947 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-12T19:48:07.014 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-12T19:48:07.027 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-12T19:48:07.077 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-12T19:48:07.140 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-12T19:48:07.263 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-12T19:48:07.305 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:19.0.0-2879.g324baf8d.el9. 39/198 2024-04-12T19:48:07.361 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el 40/198 2024-04-12T19:48:07.396 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_6 41/198 2024-04-12T19:48:07.512 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-12T19:48:07.573 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-12T19:48:07.675 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-12T19:48:07.863 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-12T19:48:07.937 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-12T19:48:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-12T19:48:08.673 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-12T19:48:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-12T19:48:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-12T19:48:08.770 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-12T19:48:08.906 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-12T19:48:08.980 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-12T19:48:09.008 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-12T19:48:09.060 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-12T19:48:09.085 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-12T19:48:09.120 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 112/198 2024-04-12T19:48:09.125 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-12T19:48:09.184 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_6 53/198 2024-04-12T19:48:09.231 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-12T19:48:09.239 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 112/198 2024-04-12T19:48:09.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 113/198 2024-04-12T19:48:09.326 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-12T19:48:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-12T19:48:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-12T19:48:09.858 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-12T19:48:09.898 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 116/198 2024-04-12T19:48:09.935 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-12T19:48:09.994 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-12T19:48:09.997 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 116/198 2024-04-12T19:48:10.025 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 117/198 2024-04-12T19:48:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-12T19:48:10.291 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-12T19:48:10.377 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-12T19:48:10.486 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-12T19:48:10.524 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-12T19:48:10.565 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-12T19:48:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-12T19:48:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-12T19:48:10.734 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-12T19:48:10.808 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-12T19:48:11.009 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-12T19:48:11.069 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-12T19:48:11.108 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-12T19:48:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-12T19:48:11.314 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-12T19:48:11.355 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-12T19:48:11.402 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-12T19:48:11.446 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-12T19:48:11.479 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-12T19:48:11.689 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-12T19:48:11.884 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-12T19:48:11.933 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-12T19:48:11.971 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-12T19:48:12.008 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-12T19:48:12.064 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-12T19:48:12.113 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-12T19:48:12.153 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-12T19:48:12.215 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-12T19:48:12.266 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-12T19:48:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-12T19:48:12.381 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-12T19:48:12.532 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-12T19:48:12.584 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-12T19:48:12.620 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-12T19:48:12.683 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-12T19:48:12.707 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:12.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-12T19:48:12.910 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-12T19:48:13.378 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:13.442 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 119/198 2024-04-12T19:48:13.616 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 119/198 2024-04-12T19:48:13.617 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-12T19:48:13.617 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-12T19:48:13.617 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:13.659 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 120/198 2024-04-12T19:48:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-12T19:48:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-12T19:48:13.992 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-12T19:48:14.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-12T19:48:14.189 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-12T19:48:14.298 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-12T19:48:14.343 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-12T19:48:14.583 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-12T19:48:14.615 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-12T19:48:14.718 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-12T19:48:15.581 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-12T19:48:15.734 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-12T19:48:17.384 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-12T19:48:17.487 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-12T19:48:17.522 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-12T19:48:17.873 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-12T19:48:17.930 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 112/198 2024-04-12T19:48:18.058 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 112/198 2024-04-12T19:48:18.089 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 113/198 2024-04-12T19:48:18.153 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-12T19:48:18.717 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-12T19:48:18.751 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 116/198 2024-04-12T19:48:18.851 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 116/198 2024-04-12T19:48:18.887 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 117/198 2024-04-12T19:48:21.542 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:21.583 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:22.246 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 118/198 2024-04-12T19:48:22.286 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 119/198 2024-04-12T19:48:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 119/198 2024-04-12T19:48:22.450 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-12T19:48:22.450 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-12T19:48:22.450 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:22.495 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 120/198 2024-04-12T19:48:26.285 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 120/198 2024-04-12T19:48:26.285 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-12T19:48:26.285 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-12T19:48:26.286 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:26.538 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 121/198 2024-04-12T19:48:26.586 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 121/198 2024-04-12T19:48:26.586 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:26.586 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-12T19:48:26.586 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 122/198 2024-04-12T19:48:28.310 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 122/198 2024-04-12T19:48:28.311 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:28.311 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-12T19:48:28.311 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T19:48:28.311 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T19:48:28.311 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:28.566 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 123/198 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 123/198 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T19:48:28.641 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:29.134 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 124/198 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 124/198 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T19:48:29.205 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:29.333 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-12T19:48:29.394 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-12T19:48:29.470 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-12T19:48:29.711 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-12T19:48:29.779 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-12T19:48:29.865 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-12T19:48:30.008 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-12T19:48:30.175 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-12T19:48:30.244 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-12T19:48:30.286 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-12T19:48:30.318 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-12T19:48:30.381 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-12T19:48:30.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-12T19:48:31.079 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 138/198 2024-04-12T19:48:31.115 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 138/198 2024-04-12T19:48:31.173 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.e 139/198 2024-04-12T19:48:31.280 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d. 140/198 2024-04-12T19:48:31.317 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 141/198 2024-04-12T19:48:32.643 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 141/198 2024-04-12T19:48:32.685 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 142/198 2024-04-12T19:48:33.790 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 142/198 2024-04-12T19:48:33.827 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 143/198 2024-04-12T19:48:33.973 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 143/198 2024-04-12T19:48:34.157 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el 144/198 2024-04-12T19:48:34.196 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 145/198 2024-04-12T19:48:34.281 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 145/198 2024-04-12T19:48:34.281 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:34.282 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-12T19:48:34.282 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T19:48:34.282 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T19:48:34.282 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:34.372 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 146/198 2024-04-12T19:48:34.412 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 146/198 2024-04-12T19:48:34.570 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 147/198 2024-04-12T19:48:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 120/198 2024-04-12T19:48:35.746 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-04-12T19:48:35.746 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-04-12T19:48:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 121/198 2024-04-12T19:48:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 121/198 2024-04-12T19:48:36.028 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:36.028 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-12T19:48:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:37.700 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 122/198 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 122/198 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T19:48:37.782 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:38.042 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 123/198 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 123/198 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T19:48:38.117 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 124/198 2024-04-12T19:48:38.715 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 124/198 2024-04-12T19:48:38.716 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:38.716 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-12T19:48:38.716 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T19:48:38.716 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T19:48:38.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:38.852 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-12T19:48:38.916 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-12T19:48:38.996 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-12T19:48:39.255 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-12T19:48:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-12T19:48:39.408 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-12T19:48:39.467 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 148/198 2024-04-12T19:48:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-12T19:48:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 149/198 2024-04-12T19:48:39.719 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-12T19:48:39.772 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-12T19:48:39.840 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-12T19:48:39.879 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-12T19:48:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-12T19:48:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-12T19:48:40.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-12T19:48:40.528 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:48:40.620 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:48:40.620 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:40.620 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-12T19:48:40.621 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T19:48:40.621 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T19:48:40.621 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:40.632 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 138/198 2024-04-12T19:48:40.671 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2879.g324ba 152/198 2024-04-12T19:48:40.675 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 138/198 2024-04-12T19:48:40.737 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.e 139/198 2024-04-12T19:48:40.742 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2879.g324ba 152/198 2024-04-12T19:48:40.743 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:40.743 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-12T19:48:40.743 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:40.805 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:48:40.858 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d. 140/198 2024-04-12T19:48:40.884 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 141/198 2024-04-12T19:48:40.886 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:48:40.887 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:40.887 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-12T19:48:40.887 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-12T19:48:40.887 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-12T19:48:40.887 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:41.272 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 154/198 2024-04-12T19:48:41.356 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 154/198 2024-04-12T19:48:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-12T19:48:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T19:48:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T19:48:41.357 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:41.429 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 155/198 2024-04-12T19:48:41.462 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 156/198 2024-04-12T19:48:41.502 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-12T19:48:41.788 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 158/198 2024-04-12T19:48:41.892 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-12T19:48:41.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-12T19:48:42.018 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-12T19:48:42.063 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-12T19:48:42.106 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_ 163/198 2024-04-12T19:48:42.149 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-12T19:48:42.202 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-12T19:48:42.252 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 141/198 2024-04-12T19:48:42.289 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-12T19:48:42.295 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 142/198 2024-04-12T19:48:42.416 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-12T19:48:42.458 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-12T19:48:43.330 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:43.380 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:43.443 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:43.450 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 142/198 2024-04-12T19:48:43.545 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 143/198 2024-04-12T19:48:43.712 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 143/198 2024-04-12T19:48:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el 144/198 2024-04-12T19:48:43.930 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 145/198 2024-04-12T19:48:44.012 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 145/198 2024-04-12T19:48:44.013 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:44.013 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-12T19:48:44.013 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T19:48:44.013 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T19:48:44.013 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:44.072 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 146/198 2024-04-12T19:48:44.112 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 146/198 2024-04-12T19:48:44.227 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 147/198 2024-04-12T19:48:44.815 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 170/198 2024-04-12T19:48:44.960 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-12T19:48:45.082 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-12T19:48:45.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-12T19:48:45.145 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-12T19:48:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-12T19:48:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-12T19:48:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-12T19:48:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-12T19:48:45.427 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-12T19:48:45.476 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-12T19:48:45.476 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-12T19:48:45.614 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-12T19:48:45.665 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-12T19:48:45.713 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-12T19:48:45.762 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-12T19:48:45.794 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-12T19:48:45.885 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-12T19:48:45.939 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-12T19:48:45.982 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-12T19:48:46.015 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-12T19:48:46.049 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-12T19:48:46.083 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-12T19:48:46.116 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-12T19:48:46.153 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:46.153 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:46.483 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:46.483 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-12T19:48:46.631 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-12T19:48:46.661 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-12T19:48:46.750 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-12T19:48:46.790 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-12T19:48:46.838 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-12T19:48:46.957 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-12T19:48:47.207 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-12T19:48:47.207 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-12T19:48:47.207 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 148/198 2024-04-12T19:48:49.458 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 149/198 2024-04-12T19:48:50.481 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-12T19:48:50.513 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:48:50.604 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:48:50.604 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:50.604 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-12T19:48:50.604 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T19:48:50.604 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T19:48:50.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:50.655 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2879.g324ba 152/198 2024-04-12T19:48:50.744 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2879.g324ba 152/198 2024-04-12T19:48:50.744 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:50.745 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-12T19:48:50.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:50.814 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:48:50.892 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:48:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-12T19:48:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-12T19:48:50.892 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-12T19:48:50.893 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:51.273 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 154/198 2024-04-12T19:48:51.349 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 154/198 2024-04-12T19:48:51.349 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T19:48:51.349 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-12T19:48:51.350 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T19:48:51.350 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T19:48:51.350 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:48:51.460 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 155/198 2024-04-12T19:48:51.501 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 156/198 2024-04-12T19:48:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-12T19:48:51.854 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 158/198 2024-04-12T19:48:51.936 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-12T19:48:51.991 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-12T19:48:52.076 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-12T19:48:52.128 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-12T19:48:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_ 163/198 2024-04-12T19:48:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-12T19:48:52.269 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-12T19:48:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-12T19:48:52.480 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-12T19:48:52.524 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 1/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 2/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 3/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 4/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2879.g324ba 5/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 6/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 7/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 8/198 2024-04-12T19:48:52.791 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 9/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 12/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 13/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 14/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_ 15/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 17/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_6 18/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 19/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 20/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el 21/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-2879.g324baf8d.el9. 22/198 2024-04-12T19:48:52.792 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_6 23/198 2024-04-12T19:48:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 24/198 2024-04-12T19:48:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 25/198 2024-04-12T19:48:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 26/198 2024-04-12T19:48:52.793 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 27/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 28/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 29/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d. 30/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 31/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 32/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 33/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el 34/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 35/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.e 36/198 2024-04-12T19:48:52.809 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 37/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 38/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 39/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-12T19:48:52.810 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-12T19:48:52.811 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-12T19:48:52.812 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-12T19:48:52.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-12T19:48:52.814 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-12T19:48:52.815 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-12T19:48:52.816 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-12T19:48:52.817 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-12T19:48:52.818 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-12T19:48:52.819 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-12T19:48:52.820 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-12T19:48:52.821 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-12T19:48:53.446 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:53.506 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:53.578 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-12T19:48:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 170/198 2024-04-12T19:48:55.175 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-12T19:48:55.297 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-12T19:48:55.353 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-12T19:48:55.379 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-12T19:48:55.413 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-12T19:48:55.448 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: acl-2.3.1-4.el9.x86_64 2024-04-12T19:48:55.459 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.460 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-12T19:48:55.461 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.462 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.463 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: jq-1.6-16.el9.x86_64 2024-04-12T19:48:55.464 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.465 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-12T19:48:55.466 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-12T19:48:55.467 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-12T19:48:55.468 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-12T19:48:55.469 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-12T19:48:55.470 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.471 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-12T19:48:55.472 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-12T19:48:55.473 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:48:55.474 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-12T19:48:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-12T19:48:55.491 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-12T19:48:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-12T19:48:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-12T19:48:55.693 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-12T19:48:55.831 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-12T19:48:55.875 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-12T19:48:55.953 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-12T19:48:55.996 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-12T19:48:56.028 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-12T19:48:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-12T19:48:56.177 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-12T19:48:56.192 DEBUG:teuthology.parallel:result is None 2024-04-12T19:48:56.225 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-12T19:48:56.258 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-12T19:48:56.291 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-12T19:48:56.325 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-12T19:48:56.358 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-12T19:48:56.395 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:56.396 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-12T19:48:56.742 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-12T19:48:56.881 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-12T19:48:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-12T19:48:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-12T19:48:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-12T19:48:57.045 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-12T19:48:57.169 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-12T19:48:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-12T19:48:57.428 INFO:teuthology.orchestra.run.smithi032.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-12T19:48:57.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 1/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 2/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 3/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 4/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2879.g324ba 5/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 6/198 2024-04-12T19:49:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 7/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 8/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 9/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 10/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 11/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 12/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 13/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 14/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_ 15/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 16/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 17/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_6 18/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86 19/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 20/198 2024-04-12T19:49:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el 21/198 2024-04-12T19:49:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:19.0.0-2879.g324baf8d.el9. 22/198 2024-04-12T19:49:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_6 23/198 2024-04-12T19:49:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 24/198 2024-04-12T19:49:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 25/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 26/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 27/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 28/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 29/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d. 30/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noa 31/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.n 32/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2879.g324 33/198 2024-04-12T19:49:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el 34/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 35/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.e 36/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 37/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 38/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 39/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-12T19:49:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-12T19:49:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-12T19:49:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-12T19:49:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-12T19:49:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-12T19:49:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-12T19:49:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-12T19:49:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-12T19:49:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 151/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 153/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-12T19:49:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-12T19:49:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-12T19:49:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-12T19:49:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-12T19:49:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-12T19:49:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-12T19:49:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-12T19:49:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-12T19:49:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-12T19:49:05.770 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: acl-2.3.1-4.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-12T19:49:05.771 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.772 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.2-8.el9.noarch 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.7.4-5.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.773 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.774 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:19.0.0-2879.g324baf8d.el9.noarch 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.4-9.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-12T19:49:05.775 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-16.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-12T19:49:05.776 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: lua-5.4.4-4.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-12T19:49:05.777 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-13.14-1.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-12T19:49:05.778 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-12T19:49:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-12T19:49:05.780 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-12T19:49:05.781 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-12T19:49:05.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-12T19:49:05.783 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:19.0.0-2879.g324baf8d.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-3.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-12T19:49:05.784 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-12T19:49:05.785 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-12T19:49:05.785 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:05.785 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-04-12T19:49:06.451 DEBUG:teuthology.parallel:result is None 2024-04-12T19:49:06.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T19:49:06.598 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T19:49:06.656 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-2879.g324baf8d.el9 2024-04-12T19:49:06.657 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2879.g324baf8d.el9 2024-04-12T19:49:06.657 INFO:teuthology.task.install:The correct ceph version 19.0.0-2879.g324baf8d is installed. 2024-04-12T19:49:06.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T19:49:06.892 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T19:49:06.965 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-2879.g324baf8d.el9 2024-04-12T19:49:06.966 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2879.g324baf8d.el9 2024-04-12T19:49:06.966 INFO:teuthology.task.install:The correct ceph version 19.0.0-2879.g324baf8d is installed. 2024-04-12T19:49:06.967 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-12T19:49:06.968 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:06.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T19:49:07.049 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:07.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T19:49:07.156 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-12T19:49:07.156 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:07.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T19:49:07.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T19:49:07.344 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:07.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T19:49:07.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T19:49:07.572 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-12T19:49:07.573 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:07.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T19:49:07.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T19:49:07.768 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:07.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T19:49:07.860 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T19:49:07.995 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-12T19:49:07.996 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:07.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T19:49:08.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T19:49:08.182 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:08.182 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T19:49:08.275 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T19:49:08.353 INFO:teuthology.run_tasks:Running task ceph... 2024-04-12T19:49:08.434 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-12T19:49:08.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-04-12T19:49:08.439 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2024-04-12T19:49:08.535 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-12T19:49:08.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-12T19:49:08.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-12T19:49:08.631 INFO:tasks.ceph:Creating extra log directories... 2024-04-12T19:49:08.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-12T19:49:08.636 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-12T19:49:08.741 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-12T19:49:08.741 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-12T19:49:08.741 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-12T19:49:08.741 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:08.741 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-12T19:49:08.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:09.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:09.084 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:09.084 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-12T19:49:09.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:09.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:09.516 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-12T19:49:09.517 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-12T19:49:09.517 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533', 'branch': 'wip-vshankar-testing-20240411.061452', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7653533', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240411.061452'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240411.061452', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7105, 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'sleep_before_teardown': 0, 'subset': '463/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240411.061452', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUEA8WnjKZ9paXPsDNdA3GYnTxMov/p6jCpaJpj6rVwsfiQF01MzE5aCFaFiMwm7KD0tEWLAYQRg8OjJZxcx2w=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCUDIFxY/GFxMBPu6HqD4URgoVn/Pp03fHbk3an3renx91zpeRxaJNCymGgVDBKijvsELAohvemX2LJoFj1EYao='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-12_03:11:38', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-12T19:49:09.517 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-12T19:49:09.523 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-12T19:49:09.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-04-12T19:49:09.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2024-04-12T19:49:09.701 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:09.701 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T19:49:09.746 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T19:49:09.746 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-12 19:48:57.896635434 +0000 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-12 19:46:00.031846498 +0000 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-12 19:46:00.031846498 +0000 2024-04-12T19:49:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-12 19:46:00.031846498 +0000 2024-04-12T19:49:09.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T19:49:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-12T19:49:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-12T19:49:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000289574 s, 1.8 MB/s 2024-04-12T19:49:09.947 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T19:49:10.007 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-12 19:48:57.897635413 +0000 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-12 19:46:00.635833725 +0000 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-12 19:46:00.635833725 +0000 2024-04-12T19:49:10.058 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-12 19:46:00.635833725 +0000 2024-04-12T19:49:10.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T19:49:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-12T19:49:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-12T19:49:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000294391 s, 1.7 MB/s 2024-04-12T19:49:10.152 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T19:49:10.209 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-04-12T19:49:10.263 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-12 19:48:57.897635413 +0000 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-12 19:46:01.194821903 +0000 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-12 19:46:01.194821903 +0000 2024-04-12T19:49:10.264 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-12 19:46:01.194821903 +0000 2024-04-12T19:49:10.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T19:49:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-12T19:49:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-12T19:49:10.360 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000274916 s, 1.9 MB/s 2024-04-12T19:49:10.365 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T19:49:10.417 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:10.467 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-12 19:48:57.898635391 +0000 2024-04-12T19:49:10.468 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-12 19:46:01.814808792 +0000 2024-04-12T19:49:10.468 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-12 19:46:01.814808792 +0000 2024-04-12T19:49:10.468 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-12 19:46:01.814808792 +0000 2024-04-12T19:49:10.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T19:49:10.561 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-12T19:49:10.561 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-12T19:49:10.561 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000243604 s, 2.1 MB/s 2024-04-12T19:49:10.565 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T19:49:10.623 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:10.624 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:10.624 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T19:49:10.681 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T19:49:10.681 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-12T19:49:10.731 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T19:49:10.731 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-12 19:48:47.634745122 +0000 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-12 19:45:57.463548191 +0000 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-12 19:45:57.464548169 +0000 2024-04-12T19:49:10.732 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-12 19:45:57.463548191 +0000 2024-04-12T19:49:10.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T19:49:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-12T19:49:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-12T19:49:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000333052 s, 1.5 MB/s 2024-04-12T19:49:10.823 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T19:49:10.877 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-12T19:49:10.926 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T19:49:10.926 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:10.926 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-12T19:49:10.926 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:10.927 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:10.927 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-12 19:48:47.634745122 +0000 2024-04-12T19:49:10.927 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-12 19:45:58.082534598 +0000 2024-04-12T19:49:10.927 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-12 19:45:58.082534598 +0000 2024-04-12T19:49:10.927 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-12 19:45:58.082534598 +0000 2024-04-12T19:49:10.927 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T19:49:11.013 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-12T19:49:11.013 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-12T19:49:11.013 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000347526 s, 1.5 MB/s 2024-04-12T19:49:11.018 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T19:49:11.073 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-12 19:48:47.635745101 +0000 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-12 19:45:58.673521619 +0000 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-12 19:45:58.673521619 +0000 2024-04-12T19:49:11.122 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-12 19:45:58.673521619 +0000 2024-04-12T19:49:11.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T19:49:11.204 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-12T19:49:11.204 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-12T19:49:11.204 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000335125 s, 1.5 MB/s 2024-04-12T19:49:11.207 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T19:49:11.261 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-12T19:49:11.311 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-12 19:48:47.636745079 +0000 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-12 19:45:59.278508333 +0000 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-12 19:45:59.278508333 +0000 2024-04-12T19:49:11.312 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-12 19:45:59.278508333 +0000 2024-04-12T19:49:11.312 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T19:49:11.401 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-12T19:49:11.401 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-12T19:49:11.401 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000347772 s, 1.5 MB/s 2024-04-12T19:49:11.405 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T19:49:11.460 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:11.460 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-12T19:49:11.460 INFO:tasks.ceph:Generating config... 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] debug client = 20 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] debug ms = 1 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] pid file = 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-12T19:49:11.462 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-12T19:49:11.462 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-12T19:49:11.462 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-12T19:49:11.463 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-12T19:49:11.463 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-12T19:49:11.463 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-12T19:49:11.463 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-12T19:49:11.463 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-12T19:49:11.464 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-12T19:49:11.465 INFO:tasks.ceph:Setting up mon.a... 2024-04-12T19:49:11.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-12T19:49:11.614 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-04-12T19:49:11.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-12T19:49:11.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-12T19:49:11.870 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2024-04-12T19:49:11.870 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-12T19:49:11.871 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:11.871 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-12T19:49:11.931 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid 47426077-081d-47b8-bfde-ce036e9b4cf8 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 47426077-081d-47b8-bfde-ce036e9b4cf8 2024-04-12T19:49:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-04-12T19:49:12.096416+0000 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-12T19:49:12.096416+0000 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 17 (quincy) 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2024-04-12T19:49:12.096 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-12T19:49:12.099 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-12T19:49:12.150 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 47426077-081d-47b8-bfde-ce036e9b4cf8... 2024-04-12T19:49:12.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-12T19:49:12.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-12T19:49:12.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-12T19:49:12.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.327 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on skipped message = False 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 47426077-081d-47b8-bfde-ce036e9b4cf8 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.328 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.329 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-12T19:49:12.330 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-12T19:49:12.331 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-12T19:49:12.332 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-12T19:49:12.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-12T19:49:12.334 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-12T19:49:12.335 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-12T19:49:12.336 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-12T19:49:12.337 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-12T19:49:12.338 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.339 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.32,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 47426077-081d-47b8-bfde-ce036e9b4cf8 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-12T19:49:12.340 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.341 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-12T19:49:12.342 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-12T19:49:12.343 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-12T19:49:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-12T19:49:12.345 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: mon down mkfs grace = 300 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-12T19:49:12.346 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-12T19:49:12.347 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-12T19:49:12.348 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-12T19:49:12.376 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-12T19:49:12.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-12T19:49:12.532 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-12T19:49:12.532 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:12.533 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-12T19:49:12.592 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:12.592 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-12T19:49:12.645 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:49:12.645 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:12.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-12T19:49:12.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-12T19:49:12.798 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:12.799 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:12.896 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:49:12.896 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:12.896 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-12T19:49:12.896 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-12T19:49:13.056 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:13.056 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:13.153 INFO:tasks.ceph:Setting up mon nodes... 2024-04-12T19:49:13.154 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-12T19:49:13.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-12T19:49:13.350 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-12T19:49:13.361 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-12T19:49:13.561 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-12T19:49:13.572 INFO:tasks.ceph:Setting up mds nodes... 2024-04-12T19:49:13.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-12T19:49:13.774 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-12T19:49:13.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-12T19:49:13.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-12T19:49:14.076 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-12T19:49:14.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-12T19:49:14.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-12T19:49:14.359 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-12T19:49:14.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-12T19:49:14.457 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-12T19:49:14.623 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-12T19:49:14.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-12T19:49:14.722 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-12T19:49:14.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-12T19:49:14.861 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-12T19:49:14.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-12T19:49:15.043 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-12T19:49:15.111 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-12T19:49:15.240 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-12T19:49:15.299 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-12T19:49:15.424 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-12T19:49:15.479 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-12T19:49:15.479 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-12T19:49:15.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-12T19:49:15.588 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:15.588 INFO:tasks.ceph:role: osd.0 2024-04-12T19:49:15.588 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:49:15.588 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:15.722 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-12T19:49:15.728 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-12T19:49:15.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-12T19:49:15.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-12T19:49:15.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-12T19:49:16.033 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:16.033 INFO:tasks.ceph:role: osd.1 2024-04-12T19:49:16.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:49:16.034 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-12T19:49:16.132 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:16.132 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:16.133 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:16.167 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-12T19:49:16.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-12T19:49:16.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-12T19:49:16.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-12T19:49:16.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-12T19:49:16.466 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:16.466 INFO:tasks.ceph:role: osd.2 2024-04-12T19:49:16.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:49:16.466 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-12T19:49:16.560 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:16.561 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:16.594 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-12T19:49:16.600 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-12T19:49:16.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-12T19:49:16.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-12T19:49:16.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-12T19:49:16.897 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:16.897 INFO:tasks.ceph:role: osd.3 2024-04-12T19:49:16.897 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:49:16.897 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-12T19:49:16.989 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:16.989 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:16.989 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:16.989 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-12T19:49:17.028 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-12T19:49:17.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-12T19:49:17.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-12T19:49:17.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:17.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.387+0000 7fd6646c2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-12T19:49:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.388+0000 7fd6646c2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-12T19:49:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.388+0000 7fd6646c2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-12T19:49:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.388+0000 7fd6646c2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-12T19:49:17.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.388+0000 7fd6646c2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-12T19:49:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:17.392+0000 7fd6646c2640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-12T19:49:20.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T19:49:20.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.878+0000 7f34c5815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-12T19:49:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.879+0000 7f34c5815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-12T19:49:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.879+0000 7f34c5815640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-12T19:49:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.879+0000 7f34c5815640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-12T19:49:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.879+0000 7f34c5815640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-12T19:49:20.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:20.881+0000 7f34c5815640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-12T19:49:23.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T19:49:23.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.139+0000 7f116640f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-12T19:49:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.140+0000 7f116640f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-12T19:49:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.140+0000 7f116640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-12T19:49:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.140+0000 7f116640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-12T19:49:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.140+0000 7f116640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-12T19:49:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:24.143+0000 7f116640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-12T19:49:27.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T19:49:27.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.632+0000 7f0380868640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-12T19:49:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.632+0000 7f0380868640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-12T19:49:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.632+0000 7f0380868640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-12T19:49:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.632+0000 7f0380868640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-12T19:49:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.632+0000 7f0380868640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-12T19:49:27.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:27.635+0000 7f0380868640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-12T19:49:30.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T19:49:30.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-12T19:49:30.794 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:30.794 INFO:tasks.ceph:role: osd.4 2024-04-12T19:49:30.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:49:30.794 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:30.891 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:31.304 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-12T19:49:31.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-12T19:49:31.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-12T19:49:31.476 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-12T19:49:31.582 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-12T19:49:31.663 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:31.663 INFO:tasks.ceph:role: osd.5 2024-04-12T19:49:31.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:49:31.663 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-12T19:49:31.748 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:31.748 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:31.748 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:31.749 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:32.106 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-12T19:49:32.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-12T19:49:32.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-12T19:49:32.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-12T19:49:32.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-12T19:49:32.378 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:32.378 INFO:tasks.ceph:role: osd.6 2024-04-12T19:49:32.378 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:49:32.378 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:32.457 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:32.458 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:32.458 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:32.458 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:32.458 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:32.843 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-12T19:49:32.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-12T19:49:32.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-12T19:49:32.984 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-12T19:49:33.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-12T19:49:33.153 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-12T19:49:33.154 INFO:tasks.ceph:role: osd.7 2024-04-12T19:49:33.154 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:49:33.154 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-12T19:49:33.231 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-12T19:49:33.231 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-12T19:49:33.231 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-12T19:49:33.232 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-12T19:49:33.641 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-12T19:49:33.648 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-12T19:49:33.648 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-12T19:49:33.821 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-12T19:49:33.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:34.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.077+0000 7fb58259a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-12T19:49:34.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.078+0000 7fb58259a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-12T19:49:34.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.078+0000 7fb58259a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-12T19:49:34.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.078+0000 7fb58259a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-12T19:49:34.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.078+0000 7fb58259a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-12T19:49:34.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:34.081+0000 7fb58259a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-12T19:49:37.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T19:49:37.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:37.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.546+0000 7f111320d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-12T19:49:37.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.547+0000 7f111320d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-12T19:49:37.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.547+0000 7f111320d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-12T19:49:37.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.547+0000 7f111320d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-12T19:49:37.548 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.547+0000 7f111320d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-12T19:49:37.550 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:37.549+0000 7f111320d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-12T19:49:39.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:39.527 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:49:40.546 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T19:49:40.643 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:40.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.780+0000 7fe23be10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-12T19:49:40.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.781+0000 7fe23be10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-12T19:49:40.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.781+0000 7fe23be10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-12T19:49:40.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.781+0000 7fe23be10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-12T19:49:40.782 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.781+0000 7fe23be10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-12T19:49:40.784 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:40.784+0000 7fe23be10640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-12T19:49:44.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-12T19:49:44.093 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:44.231 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.231+0000 7ff75c51d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-12T19:49:44.231 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.231+0000 7ff75c51d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-12T19:49:44.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.231+0000 7ff75c51d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-12T19:49:44.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.231+0000 7ff75c51d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-12T19:49:44.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.231+0000 7ff75c51d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-12T19:49:44.234 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:49:44.234+0000 7ff75c51d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-12T19:49:47.203 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-12T19:49:47.298 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-12T19:49:47.298 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-12T19:49:47.398 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-12T19:49:47.492 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-12T19:49:47.596 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-12T19:49:47.699 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-12T19:49:47.795 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-12T19:49:47.893 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:47.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-12T19:49:47.996 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:47.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-12T19:49:48.094 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.094 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-12T19:49:48.189 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-12T19:49:48.287 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-12T19:49:48.382 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.382 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-12T19:49:48.473 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-12T19:49:48.567 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.568 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-12T19:49:48.657 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:48.658 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-12T19:49:48.718 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:48.718 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-12T19:49:48.775 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.775 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-12T19:49:48.832 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:48.832 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-12T19:49:48.884 INFO:tasks.ceph:Adding keys to all mons... 2024-04-12T19:49:48.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-12T19:49:48.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-12T19:49:48.982 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2024-04-12T19:49:48.982 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA5kBlmrLL3FBAAe6dbs5ttGba1qoSrVBLpMw== 2024-04-12T19:49:48.982 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-12T19:49:48.982 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA5kBlm+J8zLhAASItkJ07VRlCtU+nHBrM6WQ== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA6kBlmEwWhBBAAicXm9WD14N+M24IV0CNtGg== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA9kBlmHjIdFxAARX/JJCf/osbmHZRsPfA9WQ== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBAkBlmwcJgNBAAT/iZHIYj+laliEVa/pJ5Wg== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBEkBlm9HtZCBAAjbA85CAONlb+W4zAWJhtxA== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBHkBlmr1GyJRAAT6SHhRDHg5kRUwht7VUGmw== 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2024-04-12T19:49:48.983 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA5kBlmyoWLIRAAFt1Wj5jPNanY8rFNEH0yRg== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA6kBlm+AuDFRAAi1aBIRh5rbqDOpn60uPFlg== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA6kBlmlAtBJRAAoou+uSYwQdHVlmLEJu8i0Q== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBOkBlmDJqyBBAAZupf7xG9Jo9iPDelxGgbtw== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBRkBlmuDCwIBAAAqfFnzUE8ONPfSv/BB6JOg== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBUkBlm70egLhAAYfk0Z4Im7bZjQ4nDSdEKXA== 2024-04-12T19:49:48.984 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBYkBlmCDnbDRAA46U5s4hIaBw4rk+7w4We7A== 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA6kBlmq9xnMxAA2e+ydW8KXvsNYpAvO4M4Xg== 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA7kBlmFu6iAhAAdZny4smaQk5UVNj+ZmZQsg== 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA7kBlmouBwDhAAVSaj/2Rp875osbza2jrMhw== 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2024-04-12T19:49:48.985 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA7kBlmYCpcGRAAfp52MuYpAUedSJCAAWVp1A== 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA5kBlmrLL3FBAAe6dbs5ttGba1qoSrVBLpMw== 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA5kBlm+J8zLhAASItkJ07VRlCtU+nHBrM6WQ== 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-12T19:49:48.986 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA6kBlmEwWhBBAAicXm9WD14N+M24IV0CNtGg== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA9kBlmHjIdFxAARX/JJCf/osbmHZRsPfA9WQ== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBAkBlmwcJgNBAAT/iZHIYj+laliEVa/pJ5Wg== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBEkBlm9HtZCBAAjbA85CAONlb+W4zAWJhtxA== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBHkBlmr1GyJRAAT6SHhRDHg5kRUwht7VUGmw== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA5kBlmyoWLIRAAFt1Wj5jPNanY8rFNEH0yRg== 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-12T19:49:48.987 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA6kBlm+AuDFRAAi1aBIRh5rbqDOpn60uPFlg== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA6kBlmlAtBJRAAoou+uSYwQdHVlmLEJu8i0Q== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBOkBlmDJqyBBAAZupf7xG9Jo9iPDelxGgbtw== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBRkBlmuDCwIBAAAqfFnzUE8ONPfSv/BB6JOg== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBUkBlm70egLhAAYfk0Z4Im7bZjQ4nDSdEKXA== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBYkBlmCDnbDRAA46U5s4hIaBw4rk+7w4We7A== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA6kBlmq9xnMxAA2e+ydW8KXvsNYpAvO4M4Xg== 2024-04-12T19:49:48.988 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2024-04-12T19:49:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA7kBlmFu6iAhAAdZny4smaQk5UVNj+ZmZQsg== 2024-04-12T19:49:48.989 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-04-12T19:49:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA7kBlmouBwDhAAVSaj/2Rp875osbza2jrMhw== 2024-04-12T19:49:48.989 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2024-04-12T19:49:48.989 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA7kBlmYCpcGRAAfp52MuYpAUedSJCAAWVp1A== 2024-04-12T19:49:48.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-12T19:49:48.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-12T19:49:49.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:49.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:49.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:49.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:49.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.613 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:49.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-12T19:49:50.064 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-12T19:49:50.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:50.219 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:50.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:50.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-12T19:49:50.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.526 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:50.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-12T19:49:51.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.433 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-12T19:49:51.738 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-12T19:49:51.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-12T19:49:51.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-12T19:49:52.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-12T19:49:52.105 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-12T19:49:52.205 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-12T19:49:52.361 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-12T19:49:52.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-12T19:49:52.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-12T19:49:52.690 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-12T19:49:52.772 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:52.781 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-12T19:49:52.831 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-12T19:49:52.832 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-12T19:49:52.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-12T19:49:52.837 INFO:tasks.ceph.mon.a:Started 2024-04-12T19:49:52.838 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-12T19:49:52.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-12T19:49:52.844 INFO:tasks.ceph.mon.b:Started 2024-04-12T19:49:52.845 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-12T19:49:52.845 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-12T19:49:52.851 INFO:tasks.ceph.mon.c:Started 2024-04-12T19:49:52.851 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-12T19:49:52.852 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-12T19:49:52.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-12T19:49:52.857 INFO:tasks.ceph.mgr.y:Started 2024-04-12T19:49:52.857 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-12T19:49:52.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-12T19:49:52.860 INFO:tasks.ceph.mgr.x:Started 2024-04-12T19:49:52.861 INFO:tasks.ceph:Setting crush tunables to default 2024-04-12T19:49:52.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-04-12T19:49:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.054+0000 7f4289800640 1 Processor -- start 2024-04-12T19:49:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.054+0000 7f4289800640 1 -- start start 2024-04-12T19:49:53.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.055+0000 7f4289800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.055+0000 7f4289800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f428415d520 con 0x7f428415a440 2024-04-12T19:49:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.055+0000 7f4289800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f428415d800 con 0x7f428406fc80 2024-04-12T19:49:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.055+0000 7f4289800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f428406e280 con 0x7f4284072860 2024-04-12T19:49:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4282000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f428406fc80 0x7f4284072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:49926/0 (socket says 172.21.15.32:49926) 2024-04-12T19:49:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 learned_addr learned my addr 172.21.15.32:0/432031775 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:49:53.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:53.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-12T19:49:53.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:53.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:53.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.056+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T19:49:53.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.257+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.258+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-12T19:49:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.258+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.258+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.258+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T19:49:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.659+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.659+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-12T19:49:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.659+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.659+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:53.660+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T19:49:54.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:54.461+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:54.462+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-12T19:49:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:54.462+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:54.462+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:54.462+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T19:49:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 msgr2=0x7f428415cef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-12T19:49:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.32:3300/0 conn(0x7f4284072860 0x7f428415cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 >> v1:172.21.15.169:6789/0 conn(0x7f428406fc80 legacy=0x7f4284072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 >> v1:172.21.15.32:6789/0 conn(0x7f428415a440 legacy=0x7f42840cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f428406e280 con 0x7f42600061d0 2024-04-12T19:49:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4260008d30 con 0x7f4260003710 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.056+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.057+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T19:49:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.258+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T19:49:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.259+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.660+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T19:49:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:56.661+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.462+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.462+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282a00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:57.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282a00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T19:49:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-12T19:49:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282000640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-12T19:49:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T19:49:57.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:57.463+0000 7f4282000640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T19:49:59.001 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:49:59.001+0000 7f5085a42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-12T19:49:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 msgr2=0x7f4260005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4260003710 0x7f4260005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 msgr2=0x7f4260008690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> v2:172.21.15.169:3300/0 conn(0x7f42600061d0 0x7f4260008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 reap_dead start 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f426000e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f428415dae0 con 0x7f4260006680 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f426000fc50 con 0x7f4260005250 2024-04-12T19:49:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.056+0000 7f427be00640 1 -- 172.21.15.32:0/432031775 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f428406e280 con 0x7f426000c1f0 2024-04-12T19:49:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.057+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f426000e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.058+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v1:172.21.15.169:6789/0 conn(0x7f4260005250 legacy=0x7f4260005700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.058+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 >> v1:172.21.15.32:6789/0 conn(0x7f4260006680 legacy=0x7f4260003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.058+0000 7f4283400640 1 -- 172.21.15.32:0/432031775 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4260010f30 con 0x7f426000c1f0 2024-04-12T19:49:59.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.058+0000 7f4281600640 1 -- 172.21.15.32:0/432031775 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485251144 0 0) 0x7f426000fc50 con 0x7f4260005250 2024-04-12T19:49:59.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.060+0000 7f4283400640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f426000e5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f428406c9d0 tx=0x7f427c02c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebec334e908bc453 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:49:59.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.060+0000 7f4281600640 1 -- 172.21.15.32:0/432031775 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c032070 con 0x7f426000c1f0 2024-04-12T19:49:59.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.060+0000 7f4281600640 1 -- 172.21.15.32:0/432031775 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c02ccb0 con 0x7f426000c1f0 2024-04-12T19:49:59.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.060+0000 7f4281600640 1 -- 172.21.15.32:0/432031775 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c036640 con 0x7f426000c1f0 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.061+0000 7f4289800640 1 -- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 msgr2=0x7f426000e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.061+0000 7f4289800640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f426000e5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f428406c9d0 tx=0x7f427c02c310 comp rx=0 tx=0).stop 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.062+0000 7f4289800640 1 -- 172.21.15.32:0/432031775 shutdown_connections 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.062+0000 7f4289800640 1 --2- 172.21.15.32:0/432031775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f426000e5e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.062+0000 7f4289800640 1 -- 172.21.15.32:0/432031775 >> 172.21.15.32:0/432031775 conn(0x7f42840b2460 msgr2=0x7f42840b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.062+0000 7f4289800640 1 -- 172.21.15.32:0/432031775 shutdown_connections 2024-04-12T19:49:59.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.062+0000 7f4289800640 1 -- 172.21.15.32:0/432031775 wait complete. 2024-04-12T19:49:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.065+0000 7f4289800640 1 Processor -- start 2024-04-12T19:49:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.065+0000 7f4289800640 1 -- start start 2024-04-12T19:49:59.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4260006680 0x7f4284139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f428413e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f428406f510 con 0x7f4260005250 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f428415ddc0 con 0x7f4260006680 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4289800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f428415cef0 con 0x7f426000c1f0 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.067+0000 7f4282a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50904/0 (socket says 172.21.15.32:50904) 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 -- 172.21.15.32:0/1239505422 learned_addr learned my addr 172.21.15.32:0/1239505422 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4283400640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f428413e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282000640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4260006680 0x7f4284139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 -- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 msgr2=0x7f428413e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f428413e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 -- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4260006680 msgr2=0x7f4284139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4260006680 0x7f4284139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.068+0000 7f4282a00640 1 -- 172.21.15.32:0/1239505422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42841413e0 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f4282a00640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4274001840 tx=0x7f4274001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427401a070 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4274013760 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4284141bb0 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274013b50 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.069+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4284142170 con 0x7f4260005250 2024-04-12T19:49:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.070+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f427401d070 con 0x7f4260005250 2024-04-12T19:49:59.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.070+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4274016790 con 0x7f4260005250 2024-04-12T19:49:59.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.071+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4284068c80 con 0x7f4260005250 2024-04-12T19:49:59.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.074+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4274013df0 con 0x7f4260005250 2024-04-12T19:49:59.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:49:59.081+0000 7f5085a42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-12T19:49:59.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.173+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4284144e70 con 0x7f4260005250 2024-04-12T19:49:59.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.176+0000 7f427aa00640 1 -- 172.21.15.32:0/1239505422 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f4274016990 con 0x7f4260005250 2024-04-12T19:49:59.177 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-04-12T19:49:59.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 msgr2=0x7f4284140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4274001840 tx=0x7f4274001d00 comp rx=0 tx=0).stop 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 shutdown_connections 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f426000c1f0 0x7f428413e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4260006680 0x7f4284139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 --2- 172.21.15.32:0/1239505422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4260005250 0x7f4284140340 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.178+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 >> 172.21.15.32:0/1239505422 conn(0x7f42840b2460 msgr2=0x7f42840b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.179+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 shutdown_connections 2024-04-12T19:49:59.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:49:59.179+0000 7f4289800640 1 -- 172.21.15.32:0/1239505422 wait complete. 2024-04-12T19:49:59.197 INFO:tasks.ceph:check_enable_crimson: False 2024-04-12T19:49:59.197 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-12T19:49:59.197 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:59.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-12T19:49:59.283 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:59.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-12T19:49:59.360 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:59.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-12T19:49:59.493 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:49:59.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-12T19:49:59.577 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:49:59.578+0000 7fd35f245040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-12T19:49:59.583 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:59.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-12T19:49:59.646 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:49:59.647+0000 7fd35f245040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-12T19:49:59.675 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:59.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-12T19:49:59.757 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:59.758 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-12T19:49:59.806 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:49:59.806+0000 7f5085a42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-12T19:49:59.879 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:49:59.878+0000 7f5085a42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-12T19:49:59.884 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:49:59.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-12T19:49:59.937 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:49:59.937+0000 7f5085a42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-12T19:49:59.963 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11 0 2024-04-12T19:50:00.007 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.007+0000 7f5085a42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-12T19:50:00.064 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.064+0000 7f5085a42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-12T19:50:00.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.201+0000 7f6d37200640 1 Processor -- start 2024-04-12T19:50:00.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.201+0000 7f6d37200640 1 -- start start 2024-04-12T19:50:00.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d37200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d300ba3e0 0x7f6d300cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d37200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d30163450 con 0x7f6d3015e9a0 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d37200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d30162e90 con 0x7f6d300cfa30 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d37200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6d300cfea0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d300ba3e0 0x7f6d300cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d300ba3e0 0x7f6d300cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52052/0 (socket says 172.21.15.169:52052) 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 -- 172.21.15.169:0/1223726826 learned_addr learned my addr 172.21.15.169:0/1223726826 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d36800640 1 --1- 172.21.15.169:0/1223726826 >> v1:172.21.15.169:6789/0 conn(0x7f6d300cfa30 0x7f6d300d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:50914/0 (socket says 172.21.15.169:50914) 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 -- 172.21.15.169:0/1223726826 >> v1:172.21.15.169:6789/0 conn(0x7f6d300cfa30 legacy=0x7f6d300d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 -- 172.21.15.169:0/1223726826 >> v1:172.21.15.32:6789/0 conn(0x7f6d3015e9a0 legacy=0x7f6d30162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:00.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.203+0000 7f6d35400640 1 -- 172.21.15.169:0/1223726826 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d30163170 con 0x7f6d300ba3e0 2024-04-12T19:50:00.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.205+0000 7f6d35400640 1 --2- 172.21.15.169:0/1223726826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300cf4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009870 tx=0x7f6d2002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f1e5e92a95bad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.205+0000 7f6d34a00640 1 -- 172.21.15.169:0/1223726826 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2002f840 con 0x7f6d300ba3e0 2024-04-12T19:50:00.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.205+0000 7f6d34a00640 1 -- 172.21.15.169:0/1223726826 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2002f9e0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.205+0000 7f6d34a00640 1 -- 172.21.15.169:0/1223726826 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2002fdb0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 -- 172.21.15.169:0/1223726826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 msgr2=0x7f6d300cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 --2- 172.21.15.169:0/1223726826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300cf4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009870 tx=0x7f6d2002ba20 comp rx=0 tx=0).stop 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 -- 172.21.15.169:0/1223726826 shutdown_connections 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 --2- 172.21.15.169:0/1223726826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300cf4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 -- 172.21.15.169:0/1223726826 >> 172.21.15.169:0/1223726826 conn(0x7f6d3006ce40 msgr2=0x7f6d3006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 -- 172.21.15.169:0/1223726826 shutdown_connections 2024-04-12T19:50:00.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.206+0000 7f6d37200640 1 -- 172.21.15.169:0/1223726826 wait complete. 2024-04-12T19:50:00.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.208+0000 7f6d37200640 1 Processor -- start 2024-04-12T19:50:00.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.208+0000 7f6d37200640 1 -- start start 2024-04-12T19:50:00.210 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.210 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 0x7f6d301608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.210 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d3015e9a0 0x7f6d30160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.210 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3015ed80 con 0x7f6d300cfa30 2024-04-12T19:50:00.210 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d300d2c00 con 0x7f6d300ba3e0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d37200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6d30161360 con 0x7f6d3015e9a0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52066/0 (socket says 172.21.15.169:52066) 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 -- 172.21.15.169:0/225063655 learned_addr learned my addr 172.21.15.169:0/225063655 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d36800640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d3015e9a0 0x7f6d30160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35400640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 0x7f6d301608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 -- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d3015e9a0 msgr2=0x7f6d30160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d3015e9a0 0x7f6d30160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 -- 172.21.15.169:0/225063655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 msgr2=0x7f6d301608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 0x7f6d301608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.209+0000 7f6d35e00640 1 -- 172.21.15.169:0/225063655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d301615b0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.211 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d35400640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 0x7f6d301608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d35e00640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3f0 tx=0x7f6d2400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24015020 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d24011830 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24011c20 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d30161850 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d30161e10 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.210+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d24018020 con 0x7f6d300ba3e0 2024-04-12T19:50:00.212 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.211+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6d24006460 con 0x7f6d300ba3e0 2024-04-12T19:50:00.213 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.209+0000 7f5085a42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-12T19:50:00.213 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.211+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d300755c0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.216+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6d2401f070 con 0x7f6d300ba3e0 2024-04-12T19:50:00.294 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.294+0000 7f5085a42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-12T19:50:00.298 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.298+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11", "id": 0} v 0) -- 0x7f6d30068ef0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.307 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.308+0000 7f6d2a000640 1 -- 172.21.15.169:0/225063655 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6d24011dc0 con 0x7f6d300ba3e0 2024-04-12T19:50:00.307 INFO:teuthology.orchestra.run.smithi169.stdout:0 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 msgr2=0x7f6d300ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b3f0 tx=0x7f6d2400b8b0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 shutdown_connections 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d3015e9a0 0x7f6d30160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d300cfa30 0x7f6d301608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 --2- 172.21.15.169:0/225063655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d300ba3e0 0x7f6d300ca9e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 >> 172.21.15.169:0/225063655 conn(0x7f6d3006ce40 msgr2=0x7f6d300d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 shutdown_connections 2024-04-12T19:50:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.309+0000 7f6d37200640 1 -- 172.21.15.169:0/225063655 wait complete. 2024-04-12T19:50:00.326 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 083a68cd-ca10-4738-9980-41cf8ed276e2 1 2024-04-12T19:50:00.326 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.326+0000 7fd35f245040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-12T19:50:00.351 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.351+0000 7f5085a42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-12T19:50:00.394 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.395+0000 7fd35f245040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-12T19:50:00.415 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.415+0000 7f5085a42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-12T19:50:00.450 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.451+0000 7fd35f245040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-12T19:50:00.473 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.473+0000 7f5085a42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-12T19:50:00.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.515+0000 7eff2b200640 1 Processor -- start 2024-04-12T19:50:00.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.515+0000 7eff2b200640 1 -- start start 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff2b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff2c15eb40 0x7eff2c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff2b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff2c0ba460 0x7eff2c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff2b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2c163400 con 0x7eff2c0d1480 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff2b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c162e40 con 0x7eff2c15eb40 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff2b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c163120 con 0x7eff2c0ba460 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff23e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff2c0ba460 0x7eff2c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff23e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff2c0ba460 0x7eff2c0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52080/0 (socket says 172.21.15.169:52080) 2024-04-12T19:50:00.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff23e00640 1 -- 172.21.15.169:0/3209880975 learned_addr learned my addr 172.21.15.169:0/3209880975 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.518 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.517+0000 7eff29e00640 1 --2- 172.21.15.169:0/3209880975 >> v2:172.21.15.32:3300/0 conn(0x7eff2c15eb40 0x7eff2c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.518 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.518+0000 7eff23e00640 1 -- 172.21.15.169:0/3209880975 >> v2:172.21.15.32:3300/0 conn(0x7eff2c15eb40 msgr2=0x7eff2c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:00.518 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.518+0000 7eff23e00640 1 --2- 172.21.15.169:0/3209880975 >> v2:172.21.15.32:3300/0 conn(0x7eff2c15eb40 0x7eff2c162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.518 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.518+0000 7eff23e00640 1 -- 172.21.15.169:0/3209880975 >> v1:172.21.15.32:6789/0 conn(0x7eff2c0d1480 legacy=0x7eff2c0b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:00.518 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.518+0000 7eff23e00640 1 -- 172.21.15.169:0/3209880975 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c0d2bc0 con 0x7eff2c0ba460 2024-04-12T19:50:00.518 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.519+0000 7fd35f245040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-12T19:50:00.519 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.519+0000 7eff23e00640 1 --2- 172.21.15.169:0/3209880975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0ba460 0x7eff2c0d0f40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0ceca0 tx=0x7eff1402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa4ffdf62a2148f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.519+0000 7eff29400640 1 -- 172.21.15.169:0/3209880975 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1402e070 con 0x7eff2c0ba460 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.519+0000 7eff29400640 1 -- 172.21.15.169:0/3209880975 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14037070 con 0x7eff2c0ba460 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.519+0000 7eff29400640 1 -- 172.21.15.169:0/3209880975 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14031a00 con 0x7eff2c0ba460 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 -- 172.21.15.169:0/3209880975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0ba460 msgr2=0x7eff2c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 --2- 172.21.15.169:0/3209880975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0ba460 0x7eff2c0d0f40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0ceca0 tx=0x7eff1402b880 comp rx=0 tx=0).stop 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 -- 172.21.15.169:0/3209880975 shutdown_connections 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 --2- 172.21.15.169:0/3209880975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0ba460 0x7eff2c0d0f40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 --2- 172.21.15.169:0/3209880975 >> v2:172.21.15.32:3300/0 conn(0x7eff2c15eb40 0x7eff2c162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 -- 172.21.15.169:0/3209880975 >> 172.21.15.169:0/3209880975 conn(0x7eff2c06d3d0 msgr2=0x7eff2c073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 -- 172.21.15.169:0/3209880975 shutdown_connections 2024-04-12T19:50:00.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.520+0000 7eff2b200640 1 -- 172.21.15.169:0/3209880975 wait complete. 2024-04-12T19:50:00.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.522+0000 7eff2b200640 1 Processor -- start 2024-04-12T19:50:00.521 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.522+0000 7eff2b200640 1 -- start start 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0ba460 0x7eff2c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c0d1480 0x7eff2c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c0d43d0 con 0x7eff2c0d1480 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c0b7860 con 0x7eff2c15eb40 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c0d4550 con 0x7eff2c0ba460 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff29e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0ba460 0x7eff2c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52084/0 (socket says 172.21.15.169:52084) 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff2a800640 1 -- 172.21.15.169:0/2219091804 learned_addr learned my addr 172.21.15.169:0/2219091804 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.523+0000 7eff23e00640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c0d1480 0x7eff2c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 -- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0ba460 msgr2=0x7eff2c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0ba460 0x7eff2c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 -- 172.21.15.169:0/2219091804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c0d1480 msgr2=0x7eff2c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c0d1480 0x7eff2c15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 -- 172.21.15.169:0/2219091804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c160230 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2a800640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c7b0 tx=0x7eff2400cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24004b60 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24004d00 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2400aa90 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c0cb9a0 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.524+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c0cbf60 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.525+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff24006b50 con 0x7eff2c15eb40 2024-04-12T19:50:00.525 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.525+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7eff240071f0 con 0x7eff2c15eb40 2024-04-12T19:50:00.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.526+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c0755c0 con 0x7eff2c15eb40 2024-04-12T19:50:00.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.528+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7eff24020070 con 0x7eff2c15eb40 2024-04-12T19:50:00.581 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.581+0000 7fd35f245040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-12T19:50:00.617 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.618+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "083a68cd-ca10-4738-9980-41cf8ed276e2", "id": 1} v 0) -- 0x7eff2c0d2360 con 0x7eff2c15eb40 2024-04-12T19:50:00.623 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.624+0000 7eff22000640 1 -- 172.21.15.169:0/2219091804 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "083a68cd-ca10-4738-9980-41cf8ed276e2", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7eff240073f0 con 0x7eff2c15eb40 2024-04-12T19:50:00.624 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 msgr2=0x7eff2c160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c7b0 tx=0x7eff2400cc70 comp rx=0 tx=0).stop 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 shutdown_connections 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c15eb40 0x7eff2c160c30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff2c0d1480 0x7eff2c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 --2- 172.21.15.169:0/2219091804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff2c0ba460 0x7eff2c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 >> 172.21.15.169:0/2219091804 conn(0x7eff2c06d3d0 msgr2=0x7eff2c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 shutdown_connections 2024-04-12T19:50:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.626+0000 7eff2b200640 1 -- 172.21.15.169:0/2219091804 wait complete. 2024-04-12T19:50:00.642 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new d78fb500-a0d8-453e-8930-c0a0dfaf6d26 2 2024-04-12T19:50:00.723 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.724+0000 7fd35f245040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-12T19:50:00.809 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.810+0000 7fd35f245040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-12T19:50:00.831 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.832+0000 7f00a9800640 1 Processor -- start 2024-04-12T19:50:00.831 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.832+0000 7f00a9800640 1 -- start start 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.833+0000 7f00a9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00a40ba460 0x7f00a40d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a9800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a4163400 con 0x7f00a415eb40 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a9800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a4162e40 con 0x7f00a40d1480 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a9800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f00a4162c20 con 0x7f00a40ba460 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00a40ba460 0x7f00a40d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.833 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00a40ba460 0x7f00a40d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52094/0 (socket says 172.21.15.169:52094) 2024-04-12T19:50:00.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 -- 172.21.15.169:0/3528226999 learned_addr learned my addr 172.21.15.169:0/3528226999 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 -- 172.21.15.169:0/3528226999 >> v1:172.21.15.169:6789/0 conn(0x7f00a40d1480 legacy=0x7f00a40d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 -- 172.21.15.169:0/3528226999 >> v1:172.21.15.32:6789/0 conn(0x7f00a415eb40 legacy=0x7f00a4162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.834+0000 7f00a3400640 1 -- 172.21.15.169:0/3528226999 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a4163120 con 0x7f00a40ba460 2024-04-12T19:50:00.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.835+0000 7f00a3400640 1 --2- 172.21.15.169:0/3528226999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a40d0f40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0090009870 tx=0x7f009002b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4f9f17e37118db server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a2a00640 1 -- 172.21.15.169:0/3528226999 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090035020 con 0x7f00a40ba460 2024-04-12T19:50:00.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a2a00640 1 -- 172.21.15.169:0/3528226999 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009002d040 con 0x7f00a40ba460 2024-04-12T19:50:00.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a2a00640 1 -- 172.21.15.169:0/3528226999 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090031a80 con 0x7f00a40ba460 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a9800640 1 -- 172.21.15.169:0/3528226999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 msgr2=0x7f00a40d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a9800640 1 --2- 172.21.15.169:0/3528226999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a40d0f40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0090009870 tx=0x7f009002b900 comp rx=0 tx=0).stop 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a9800640 1 -- 172.21.15.169:0/3528226999 shutdown_connections 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a9800640 1 --2- 172.21.15.169:0/3528226999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a40d0f40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.836+0000 7f00a9800640 1 -- 172.21.15.169:0/3528226999 >> 172.21.15.169:0/3528226999 conn(0x7f00a406d3d0 msgr2=0x7f00a4073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.837+0000 7f00a9800640 1 -- 172.21.15.169:0/3528226999 shutdown_connections 2024-04-12T19:50:00.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.837+0000 7f00a9800640 1 -- 172.21.15.169:0/3528226999 wait complete. 2024-04-12T19:50:00.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.838+0000 7f00a9800640 1 Processor -- start 2024-04-12T19:50:00.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.838+0000 7f00a9800640 1 -- start start 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 0x7f00a40ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a415eb40 0x7f00a40b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40b7860 con 0x7f00a415eb40 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40d2ea0 con 0x7f00a40ba460 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f00a40d3ad0 con 0x7f00a40d1480 2024-04-12T19:50:00.840 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.839+0000 7f00a3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 0x7f00a40ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52110/0 (socket says 172.21.15.169:52110) 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 -- 172.21.15.169:0/2195867193 learned_addr learned my addr 172.21.15.169:0/2195867193 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a8e00640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a415eb40 0x7f00a40b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 -- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 msgr2=0x7f00a40ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 0x7f00a40ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 -- 172.21.15.169:0/2195867193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a415eb40 msgr2=0x7f00a40b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a415eb40 0x7f00a40b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.841 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 -- 172.21.15.169:0/2195867193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40c8c80 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3400640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 0x7f00a40ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.840+0000 7f00a3e00640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f009400c3f0 tx=0x7f009400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094011790 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0094017070 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094011b90 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a40c8f60 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a40c9520 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f009401c460 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.841+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0094011d30 con 0x7f00a40ba460 2024-04-12T19:50:00.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.842+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a40755c0 con 0x7f00a40ba460 2024-04-12T19:50:00.843 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.844+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0094012600 con 0x7f00a40ba460 2024-04-12T19:50:00.869 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.869+0000 7fd35f245040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-12T19:50:00.922 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.923+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d78fb500-a0d8-453e-8930-c0a0dfaf6d26", "id": 2} v 0) -- 0x7f00a40b6f90 con 0x7f00a40ba460 2024-04-12T19:50:00.929 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.930+0000 7f009be00640 1 -- 172.21.15.169:0/2195867193 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d78fb500-a0d8-453e-8930-c0a0dfaf6d26", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f00940128e0 con 0x7f00a40ba460 2024-04-12T19:50:00.929 INFO:teuthology.orchestra.run.smithi169.stdout:2 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 msgr2=0x7f00a4153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f009400c3f0 tx=0x7f009400c8b0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 shutdown_connections 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a415eb40 0x7f00a40b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00a40d1480 0x7f00a40ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 --2- 172.21.15.169:0/2195867193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00a40ba460 0x7f00a4153ca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 >> 172.21.15.169:0/2195867193 conn(0x7f00a406d3d0 msgr2=0x7f00a40cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.931+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 shutdown_connections 2024-04-12T19:50:00.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:00.932+0000 7f00a9800640 1 -- 172.21.15.169:0/2195867193 wait complete. 2024-04-12T19:50:00.934 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.935+0000 7fd35f245040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-12T19:50:00.948 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e6e2c7ff-5821-4435-81bf-5d3265e5a5f5 3 2024-04-12T19:50:00.949 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:00.949+0000 7f5085a42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-12T19:50:00.996 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:00.996+0000 7fd35f245040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-12T19:50:01.119 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.119+0000 7f5085a42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-12T19:50:01.140 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.141+0000 7f01a5200640 1 Processor -- start 2024-04-12T19:50:01.141 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.141+0000 7f01a5200640 1 -- start start 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f01a5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01a00ba460 0x7f01a00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f01a5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a00d1480 0x7f01a00d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f01a5200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a0163400 con 0x7f01a015eb40 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f01a5200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01a0162c20 con 0x7f01a00d1480 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f01a5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f01a0162e40 con 0x7f01a00ba460 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f019ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01a00ba460 0x7f01a00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f019ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01a00ba460 0x7f01a00d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52112/0 (socket says 172.21.15.169:52112) 2024-04-12T19:50:01.143 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f019ea00640 1 -- 172.21.15.169:0/1147130720 learned_addr learned my addr 172.21.15.169:0/1147130720 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.143+0000 7f019fe00640 1 --2- 172.21.15.169:0/1147130720 >> v2:172.21.15.32:3300/0 conn(0x7f01a00d1480 0x7f01a00d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.144+0000 7f019ea00640 1 -- 172.21.15.169:0/1147130720 >> v2:172.21.15.32:3300/0 conn(0x7f01a00d1480 msgr2=0x7f01a00d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.144+0000 7f019ea00640 1 --2- 172.21.15.169:0/1147130720 >> v2:172.21.15.32:3300/0 conn(0x7f01a00d1480 0x7f01a00d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.144+0000 7f019ea00640 1 -- 172.21.15.169:0/1147130720 >> v1:172.21.15.32:6789/0 conn(0x7f01a015eb40 legacy=0x7f01a0162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.144+0000 7f019ea00640 1 -- 172.21.15.169:0/1147130720 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a0163120 con 0x7f01a00ba460 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.145+0000 7f019ea00640 1 --2- 172.21.15.169:0/1147130720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a00ba460 0x7f01a00d0f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01a00ceca0 tx=0x7f018c02b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905089d96dd484f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.144 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.145+0000 7f019e000640 1 -- 172.21.15.169:0/1147130720 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c02e070 con 0x7f01a00ba460 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.145+0000 7f019e000640 1 -- 172.21.15.169:0/1147130720 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c037070 con 0x7f01a00ba460 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.145+0000 7f019e000640 1 -- 172.21.15.169:0/1147130720 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c031a00 con 0x7f01a00ba460 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 -- 172.21.15.169:0/1147130720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a00ba460 msgr2=0x7f01a00d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 --2- 172.21.15.169:0/1147130720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a00ba460 0x7f01a00d0f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f01a00ceca0 tx=0x7f018c02b880 comp rx=0 tx=0).stop 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 -- 172.21.15.169:0/1147130720 shutdown_connections 2024-04-12T19:50:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 --2- 172.21.15.169:0/1147130720 >> v2:172.21.15.32:3300/0 conn(0x7f01a00d1480 0x7f01a00d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 --2- 172.21.15.169:0/1147130720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a00ba460 0x7f01a00d0f40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 -- 172.21.15.169:0/1147130720 >> 172.21.15.169:0/1147130720 conn(0x7f01a006d3d0 msgr2=0x7f01a0073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 -- 172.21.15.169:0/1147130720 shutdown_connections 2024-04-12T19:50:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.146+0000 7f01a5200640 1 -- 172.21.15.169:0/1147130720 wait complete. 2024-04-12T19:50:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.148+0000 7f01a5200640 1 Processor -- start 2024-04-12T19:50:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.148+0000 7f01a5200640 1 -- start start 2024-04-12T19:50:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a00ba460 0x7f01a01f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 0x7f01a01f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b7860 con 0x7f01a00ba460 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00d2ea0 con 0x7f01a015eb40 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f01a5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00d3900 con 0x7f01a00d1480 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f019ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 0x7f01a01f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f019fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f019fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52122/0 (socket says 172.21.15.169:52122) 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f019fe00640 1 -- 172.21.15.169:0/4286770775 learned_addr learned my addr 172.21.15.169:0/4286770775 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:01.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.149+0000 7f019ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 0x7f01a01f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46142/0 (socket says 172.21.15.169:46142) 2024-04-12T19:50:01.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 -- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 msgr2=0x7f01a01f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 0x7f01a01f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 -- 172.21.15.169:0/4286770775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a00ba460 msgr2=0x7f01a01f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a00ba460 0x7f01a01f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 -- 172.21.15.169:0/4286770775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01e9270 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f019fe00640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0198007e70 tx=0x7f019800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198014620 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0198019070 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198014a20 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a01e9550 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.150+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01e9b10 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.151+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0198014bc0 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.151+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f019801e8f0 con 0x7f01a015eb40 2024-04-12T19:50:01.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.151+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a00755c0 con 0x7f01a015eb40 2024-04-12T19:50:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.155+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f019801e460 con 0x7f01a015eb40 2024-04-12T19:50:01.233 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.234+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6e2c7ff-5821-4435-81bf-5d3265e5a5f5", "id": 3} v 0) -- 0x7f01a00d2360 con 0x7f01a015eb40 2024-04-12T19:50:01.239 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.240+0000 7f0197400640 1 -- 172.21.15.169:0/4286770775 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6e2c7ff-5821-4435-81bf-5d3265e5a5f5", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f019801e6e0 con 0x7f01a015eb40 2024-04-12T19:50:01.239 INFO:teuthology.orchestra.run.smithi169.stdout:3 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.241+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 msgr2=0x7f01a01f8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.241+0000 7f01a5200640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0198007e70 tx=0x7f019800e4b0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 shutdown_connections 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01a015eb40 0x7f01a01f8470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01a00d1480 0x7f01a01f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 --2- 172.21.15.169:0/4286770775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a00ba460 0x7f01a01f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 >> 172.21.15.169:0/4286770775 conn(0x7f01a006d3d0 msgr2=0x7f01a00cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 shutdown_connections 2024-04-12T19:50:01.241 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.242+0000 7f01a5200640 1 -- 172.21.15.169:0/4286770775 wait complete. 2024-04-12T19:50:01.253 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.253+0000 7f5085a42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-12T19:50:01.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new b881395c-671e-4789-a57f-d80d3bdc3ab8 4 2024-04-12T19:50:01.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.309+0000 7f5085a42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-12T19:50:01.444 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.444+0000 7f5085a42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-12T19:50:01.452 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.452+0000 7fb579c00640 1 Processor -- start 2024-04-12T19:50:01.452 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.453+0000 7fb579c00640 1 -- start start 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.454+0000 7fb579c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb57415eb40 0x7fb574162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.454+0000 7fb579c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.454+0000 7fb579c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb574163400 con 0x7fb5740ba460 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.454+0000 7fb579c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb574162e40 con 0x7fb57415eb40 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.454+0000 7fb579c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb574163120 con 0x7fb5740d1650 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.455 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46148/0 (socket says 172.21.15.169:46148) 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 -- 172.21.15.169:0/2772319980 learned_addr learned my addr 172.21.15.169:0/2772319980 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb573e00640 1 --2- 172.21.15.169:0/2772319980 >> v2:172.21.15.32:3300/0 conn(0x7fb57415eb40 0x7fb574162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 -- 172.21.15.169:0/2772319980 >> v2:172.21.15.32:3300/0 conn(0x7fb57415eb40 msgr2=0x7fb574162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 --2- 172.21.15.169:0/2772319980 >> v2:172.21.15.32:3300/0 conn(0x7fb57415eb40 0x7fb574162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.455+0000 7fb579200640 1 -- 172.21.15.169:0/2772319980 >> v1:172.21.15.32:6789/0 conn(0x7fb5740ba460 legacy=0x7fb5740d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:01.456 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.456+0000 7fb579200640 1 -- 172.21.15.169:0/2772319980 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5740d2bc0 con 0x7fb5740d1650 2024-04-12T19:50:01.457 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.457+0000 7fb579200640 1 --2- 172.21.15.169:0/2772319980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009e50 tx=0x7fb56c02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc893c49f999aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.457+0000 7fb573400640 1 -- 172.21.15.169:0/2772319980 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c02c9e0 con 0x7fb5740d1650 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.457+0000 7fb573400640 1 -- 172.21.15.169:0/2772319980 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c02cb80 con 0x7fb5740d1650 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.457+0000 7fb573400640 1 -- 172.21.15.169:0/2772319980 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0343f0 con 0x7fb5740d1650 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 -- 172.21.15.169:0/2772319980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 msgr2=0x7fb5740d33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 --2- 172.21.15.169:0/2772319980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009e50 tx=0x7fb56c02be40 comp rx=0 tx=0).stop 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 -- 172.21.15.169:0/2772319980 shutdown_connections 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 --2- 172.21.15.169:0/2772319980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb5740d33c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 --2- 172.21.15.169:0/2772319980 >> v2:172.21.15.32:3300/0 conn(0x7fb57415eb40 0x7fb574162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 -- 172.21.15.169:0/2772319980 >> 172.21.15.169:0/2772319980 conn(0x7fb57406d3d0 msgr2=0x7fb574073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 -- 172.21.15.169:0/2772319980 shutdown_connections 2024-04-12T19:50:01.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.458+0000 7fb579c00640 1 -- 172.21.15.169:0/2772319980 wait complete. 2024-04-12T19:50:01.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.460+0000 7fb579c00640 1 Processor -- start 2024-04-12T19:50:01.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.460+0000 7fb579c00640 1 -- start start 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5740ba460 0x7fb5740ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 0x7fb5740b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb574162a50 con 0x7fb5740ba460 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5740b7860 con 0x7fb57415eb40 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb574162bd0 con 0x7fb5740d1650 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb56be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb579200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 0x7fb5740b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb56be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46158/0 (socket says 172.21.15.169:46158) 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb56be00640 1 -- 172.21.15.169:0/294535037 learned_addr learned my addr 172.21.15.169:0/294535037 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.461+0000 7fb573e00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5740ba460 0x7fb5740ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.462 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 -- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 msgr2=0x7fb5740b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 0x7fb5740b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 -- 172.21.15.169:0/294535037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5740ba460 msgr2=0x7fb5740ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5740ba460 0x7fb5740ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 -- 172.21.15.169:0/294535037 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5740cd310 con 0x7fb5740d1650 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb579200640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 0x7fb5740b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb56be00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c400 tx=0x7fb55c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011790 con 0x7fb5740d1650 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c017070 con 0x7fb5740d1650 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c011bb0 con 0x7fb5740d1650 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.462+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5740cd5f0 con 0x7fb5740d1650 2024-04-12T19:50:01.463 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.463+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5740cdbb0 con 0x7fb5740d1650 2024-04-12T19:50:01.464 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.463+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb55c01c460 con 0x7fb5740d1650 2024-04-12T19:50:01.464 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.463+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb55c011d50 con 0x7fb5740d1650 2024-04-12T19:50:01.464 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.464+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5740755c0 con 0x7fb5740d1650 2024-04-12T19:50:01.466 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.467+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb55c0126a0 con 0x7fb5740d1650 2024-04-12T19:50:01.486 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:01.487+0000 7fd35f245040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-12T19:50:01.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.547+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b881395c-671e-4789-a57f-d80d3bdc3ab8", "id": 4} v 0) -- 0x7fb5740b9de0 con 0x7fb5740d1650 2024-04-12T19:50:01.553 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.553+0000 7fb570c00640 1 -- 172.21.15.169:0/294535037 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b881395c-671e-4789-a57f-d80d3bdc3ab8", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fb55c0128e0 con 0x7fb5740d1650 2024-04-12T19:50:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:4 2024-04-12T19:50:01.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 msgr2=0x7fb574149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c400 tx=0x7fb55c00c8c0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.554 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 shutdown_connections 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57415eb40 0x7fb5740b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5740d1650 0x7fb574149ab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 --2- 172.21.15.169:0/294535037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5740ba460 0x7fb5740ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 >> 172.21.15.169:0/294535037 conn(0x7fb57406d3d0 msgr2=0x7fb574073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 shutdown_connections 2024-04-12T19:50:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.555+0000 7fb579c00640 1 -- 172.21.15.169:0/294535037 wait complete. 2024-04-12T19:50:01.562 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.562+0000 7f5085a42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-12T19:50:01.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new f115ff76-a67b-4003-bd9f-758b919db1b1 5 2024-04-12T19:50:01.659 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:01.660+0000 7fd35f245040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-12T19:50:01.749 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.748+0000 7f5085a42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-12T19:50:01.760 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.761+0000 7f13cb200640 1 Processor -- start 2024-04-12T19:50:01.761 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.761+0000 7f13cb200640 1 -- start start 2024-04-12T19:50:01.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 0x7f13cc162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13cb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc163400 con 0x7f13cc0ba460 2024-04-12T19:50:01.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13cb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc162e40 con 0x7f13cc0d1650 2024-04-12T19:50:01.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc163120 con 0x7f13cc15eb40 2024-04-12T19:50:01.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13c9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 0x7f13cc162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13ca800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f13cc0d1650 0x7f13cc0d33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:50928/0 (socket says 172.21.15.169:50928) 2024-04-12T19:50:01.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.763+0000 7f13ca800640 1 -- 172.21.15.169:0/2269619430 learned_addr learned my addr 172.21.15.169:0/2269619430 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.764+0000 7f13c9e00640 1 -- 172.21.15.169:0/2269619430 >> v1:172.21.15.169:6789/0 conn(0x7f13cc0d1650 legacy=0x7f13cc0d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.764+0000 7f13c9e00640 1 -- 172.21.15.169:0/2269619430 >> v1:172.21.15.32:6789/0 conn(0x7f13cc0ba460 legacy=0x7f13cc0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.764+0000 7f13c9e00640 1 -- 172.21.15.169:0/2269619430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc0d2bc0 con 0x7f13cc15eb40 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.764+0000 7f13c9400640 1 -- 172.21.15.169:0/2269619430 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449811997 0 0) 0x7f13cc162e40 con 0x7f13cc0d1650 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13c9e00640 1 --2- 172.21.15.169:0/2269619430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 0x7f13cc162510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0097d0 tx=0x7f13bc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b967096f525358 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13c9400640 1 -- 172.21.15.169:0/2269619430 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035020 con 0x7f13cc15eb40 2024-04-12T19:50:01.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13c9400640 1 -- 172.21.15.169:0/2269619430 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc031800 con 0x7f13cc15eb40 2024-04-12T19:50:01.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13c9400640 1 -- 172.21.15.169:0/2269619430 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc031bd0 con 0x7f13cc15eb40 2024-04-12T19:50:01.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13cb200640 1 -- 172.21.15.169:0/2269619430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 msgr2=0x7f13cc162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.765+0000 7f13cb200640 1 --2- 172.21.15.169:0/2269619430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 0x7f13cc162510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0097d0 tx=0x7f13bc02b8a0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.766+0000 7f13cb200640 1 -- 172.21.15.169:0/2269619430 shutdown_connections 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.766+0000 7f13cb200640 1 --2- 172.21.15.169:0/2269619430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc15eb40 0x7f13cc162510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.766+0000 7f13cb200640 1 -- 172.21.15.169:0/2269619430 >> 172.21.15.169:0/2269619430 conn(0x7f13cc06d3d0 msgr2=0x7f13cc073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.766+0000 7f13cb200640 1 -- 172.21.15.169:0/2269619430 shutdown_connections 2024-04-12T19:50:01.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.766+0000 7f13cb200640 1 -- 172.21.15.169:0/2269619430 wait complete. 2024-04-12T19:50:01.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.767+0000 7f13cb200640 1 Processor -- start 2024-04-12T19:50:01.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.767+0000 7f13cb200640 1 -- start start 2024-04-12T19:50:01.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13cc0d1650 0x7f13cc0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 0x7f13cc0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc162a50 con 0x7f13cc0d1650 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0b7860 con 0x7f13cc15eb40 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc162bd0 con 0x7f13cc0ba460 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13c9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13c9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46174/0 (socket says 172.21.15.169:46174) 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.768+0000 7f13c9e00640 1 -- 172.21.15.169:0/855647027 learned_addr learned my addr 172.21.15.169:0/855647027 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13ca800640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 0x7f13cc0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c3e00640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13cc0d1650 0x7f13cc0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 -- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 msgr2=0x7f13cc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 0x7f13cc0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 -- 172.21.15.169:0/855647027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13cc0d1650 msgr2=0x7f13cc0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13cc0d1650 0x7f13cc0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 -- 172.21.15.169:0/855647027 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc160530 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13ca800640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 0x7f13cc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.769+0000 7f13c9e00640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098f0 tx=0x7f13bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc040070 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc03e070 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc1f46e0 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035040 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1f4ca0 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f13bc038020 con 0x7f13cc0ba460 2024-04-12T19:50:01.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.770+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f13bc045cd0 con 0x7f13cc0ba460 2024-04-12T19:50:01.772 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.771+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc0755c0 con 0x7f13cc0ba460 2024-04-12T19:50:01.772 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.773+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f13bc045460 con 0x7f13cc0ba460 2024-04-12T19:50:01.800 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:01.801+0000 7fd35f245040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-12T19:50:01.801 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.801+0000 7f5085a42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-12T19:50:01.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.853+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f115ff76-a67b-4003-bd9f-758b919db1b1", "id": 5} v 0) -- 0x7f13cc0d2360 con 0x7f13cc0ba460 2024-04-12T19:50:01.857 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:01.858+0000 7fd35f245040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-12T19:50:01.858 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.859+0000 7f13c2000640 1 -- 172.21.15.169:0/855647027 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f115ff76-a67b-4003-bd9f-758b919db1b1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f13bc045640 con 0x7f13cc0ba460 2024-04-12T19:50:01.858 INFO:teuthology.orchestra.run.smithi169.stdout:5 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.860+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 msgr2=0x7f13cc15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.860+0000 7f13cb200640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098f0 tx=0x7f13bc0046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 shutdown_connections 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13cc15eb40 0x7f13cc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13cc0d1650 0x7f13cc0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 --2- 172.21.15.169:0/855647027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13cc0ba460 0x7f13cc15bd40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:01.860 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 >> 172.21.15.169:0/855647027 conn(0x7f13cc06d3d0 msgr2=0x7f13cc0b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:01.861 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 shutdown_connections 2024-04-12T19:50:01.861 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:01.861+0000 7f13cb200640 1 -- 172.21.15.169:0/855647027 wait complete. 2024-04-12T19:50:01.876 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e4d48e5d-0727-437c-bdae-552adb15888b 6 2024-04-12T19:50:01.988 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:01.987+0000 7f5085a42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-12T19:50:01.995 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:01.996+0000 7fd35f245040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-12T19:50:02.068 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.069+0000 7f0303200640 1 Processor -- start 2024-04-12T19:50:02.068 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.069+0000 7f0303200640 1 -- start start 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.070+0000 7f0303200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.070+0000 7f0303200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f02fc06be60 0x7f02fc06c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.070+0000 7f0303200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02fc15ee40 con 0x7f02fc0731f0 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0303200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f02fc06c800 con 0x7f02fc06be60 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0303200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02fc0d1880 con 0x7f02fc15ea60 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46176/0 (socket says 172.21.15.169:46176) 2024-04-12T19:50:02.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f02fc06be60 0x7f02fc06c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 -- 172.21.15.169:0/1841728249 learned_addr learned my addr 172.21.15.169:0/1841728249 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 -- 172.21.15.169:0/1841728249 >> v2:172.21.15.169:3300/0 conn(0x7f02fc06be60 msgr2=0x7f02fc06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 --2- 172.21.15.169:0/1841728249 >> v2:172.21.15.169:3300/0 conn(0x7f02fc06be60 0x7f02fc06c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 -- 172.21.15.169:0/1841728249 >> v1:172.21.15.32:6789/0 conn(0x7f02fc0731f0 legacy=0x7f02fc073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.071+0000 7f0301e00640 1 -- 172.21.15.169:0/1841728249 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc073f50 con 0x7f02fc15ea60 2024-04-12T19:50:02.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.072+0000 7f0301e00640 1 --2- 172.21.15.169:0/1841728249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009870 tx=0x7f02f002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ca4579e60639a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.072+0000 7f0300a00640 1 -- 172.21.15.169:0/1841728249 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035020 con 0x7f02fc15ea60 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.072+0000 7f0300a00640 1 -- 172.21.15.169:0/1841728249 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f0031640 con 0x7f02fc15ea60 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.072+0000 7f0300a00640 1 -- 172.21.15.169:0/1841728249 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f002d040 con 0x7f02fc15ea60 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 -- 172.21.15.169:0/1841728249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 msgr2=0x7f02fc0d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 --2- 172.21.15.169:0/1841728249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009870 tx=0x7f02f002b8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 -- 172.21.15.169:0/1841728249 shutdown_connections 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 --2- 172.21.15.169:0/1841728249 >> v2:172.21.15.169:3300/0 conn(0x7f02fc06be60 0x7f02fc06c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 --2- 172.21.15.169:0/1841728249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc0d1d10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 -- 172.21.15.169:0/1841728249 >> 172.21.15.169:0/1841728249 conn(0x7f02fc06ca40 msgr2=0x7f02fc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 -- 172.21.15.169:0/1841728249 shutdown_connections 2024-04-12T19:50:02.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.073+0000 7f0303200640 1 -- 172.21.15.169:0/1841728249 wait complete. 2024-04-12T19:50:02.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.075+0000 7f0303200640 1 Processor -- start 2024-04-12T19:50:02.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.075+0000 7f0303200640 1 -- start start 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 0x7f02fc15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc0d1510 con 0x7f02fc0731f0 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc0d1230 con 0x7f02fc06be60 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0303200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02fc0d2250 con 0x7f02fc15ea60 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0301e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0301e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52160/0 (socket says 172.21.15.169:52160) 2024-04-12T19:50:02.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0301e00640 1 -- 172.21.15.169:0/3157077430 learned_addr learned my addr 172.21.15.169:0/3157077430 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.076+0000 7f0302800640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301400640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 0x7f02fc15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 -- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 msgr2=0x7f02fc15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 -- 172.21.15.169:0/3157077430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 msgr2=0x7f02fc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 0x7f02fc15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 -- 172.21.15.169:0/3157077430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc157da0 con 0x7f02fc06be60 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301400640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 0x7f02fc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0301e00640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02f002b930 tx=0x7f02f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0040070 con 0x7f02fc06be60 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f00399f0 con 0x7f02fc06be60 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0035040 con 0x7f02fc06be60 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.077+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc158080 con 0x7f02fc06be60 2024-04-12T19:50:02.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.078+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc158640 con 0x7f02fc06be60 2024-04-12T19:50:02.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.078+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02f004b060 con 0x7f02fc06be60 2024-04-12T19:50:02.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.078+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f02f0039e30 con 0x7f02fc06be60 2024-04-12T19:50:02.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.079+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc0755c0 con 0x7f02fc06be60 2024-04-12T19:50:02.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.081+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f02f004d7e0 con 0x7f02fc06be60 2024-04-12T19:50:02.116 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:02.117+0000 7fd35f245040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-12T19:50:02.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.159+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e4d48e5d-0727-437c-bdae-552adb15888b", "id": 6} v 0) -- 0x7f02fc06e040 con 0x7f02fc06be60 2024-04-12T19:50:02.164 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.164+0000 7f02f6000640 1 -- 172.21.15.169:0/3157077430 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e4d48e5d-0727-437c-bdae-552adb15888b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f02f004d9c0 con 0x7f02fc06be60 2024-04-12T19:50:02.164 INFO:teuthology.orchestra.run.smithi169.stdout:6 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 msgr2=0x7f02fc155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f02f002b930 tx=0x7f02f00046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 shutdown_connections 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02fc15ea60 0x7f02fc15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02fc0731f0 0x7f02fc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 --2- 172.21.15.169:0/3157077430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02fc06be60 0x7f02fc155db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 >> 172.21.15.169:0/3157077430 conn(0x7f02fc06ca40 msgr2=0x7f02fc0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.166 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 shutdown_connections 2024-04-12T19:50:02.166 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.166+0000 7f0303200640 1 -- 172.21.15.169:0/3157077430 wait complete. 2024-04-12T19:50:02.183 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 7661f002-989b-4c53-8297-888c0c4529d2 7 2024-04-12T19:50:02.308 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:02.308+0000 7fd35f245040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-12T19:50:02.362 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:02.363+0000 7fd35f245040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-12T19:50:02.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.366+0000 7ffa67200640 1 Processor -- start 2024-04-12T19:50:02.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.366+0000 7ffa67200640 1 -- start start 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa67200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa67200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa680d1480 0x7ffa680d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa67200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa68163400 con 0x7ffa6815eb40 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa67200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffa68162c20 con 0x7ffa680d1480 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa67200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa68162e40 con 0x7ffa680ba460 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46198/0 (socket says 172.21.15.169:46198) 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 -- 172.21.15.169:0/4149034929 learned_addr learned my addr 172.21.15.169:0/4149034929 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa66800640 1 --2- 172.21.15.169:0/4149034929 >> v2:172.21.15.32:3300/0 conn(0x7ffa680d1480 0x7ffa680d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 -- 172.21.15.169:0/4149034929 >> v2:172.21.15.32:3300/0 conn(0x7ffa680d1480 msgr2=0x7ffa680d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 --2- 172.21.15.169:0/4149034929 >> v2:172.21.15.32:3300/0 conn(0x7ffa680d1480 0x7ffa680d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 -- 172.21.15.169:0/4149034929 >> v1:172.21.15.32:6789/0 conn(0x7ffa6815eb40 legacy=0x7ffa68162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:02.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.368+0000 7ffa65400640 1 -- 172.21.15.169:0/4149034929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68163120 con 0x7ffa680ba460 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.369+0000 7ffa65400640 1 --2- 172.21.15.169:0/4149034929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009870 tx=0x7ffa5402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206bda470fe01214 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.370+0000 7ffa64a00640 1 -- 172.21.15.169:0/4149034929 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540317c0 con 0x7ffa680ba460 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.370+0000 7ffa64a00640 1 -- 172.21.15.169:0/4149034929 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54037070 con 0x7ffa680ba460 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.370+0000 7ffa64a00640 1 -- 172.21.15.169:0/4149034929 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54031b80 con 0x7ffa680ba460 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.370+0000 7ffa67200640 1 -- 172.21.15.169:0/4149034929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 msgr2=0x7ffa680d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.370+0000 7ffa67200640 1 --2- 172.21.15.169:0/4149034929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009870 tx=0x7ffa5402b860 comp rx=0 tx=0).stop 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 -- 172.21.15.169:0/4149034929 shutdown_connections 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 --2- 172.21.15.169:0/4149034929 >> v2:172.21.15.32:3300/0 conn(0x7ffa680d1480 0x7ffa680d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 --2- 172.21.15.169:0/4149034929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680ba460 0x7ffa680d0f40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 -- 172.21.15.169:0/4149034929 >> 172.21.15.169:0/4149034929 conn(0x7ffa6806d3d0 msgr2=0x7ffa68073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 -- 172.21.15.169:0/4149034929 shutdown_connections 2024-04-12T19:50:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.371+0000 7ffa67200640 1 -- 172.21.15.169:0/4149034929 wait complete. 2024-04-12T19:50:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.373+0000 7ffa67200640 1 Processor -- start 2024-04-12T19:50:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.373+0000 7ffa67200640 1 -- start start 2024-04-12T19:50:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa680ba460 0x7ffa68153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6815eb40 0x7ffa6815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680b7860 con 0x7ffa680ba460 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680d2ea0 con 0x7ffa6815eb40 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa67200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa680d3900 con 0x7ffa680d1480 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa66800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6815eb40 0x7ffa6815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:46214/0 (socket says 172.21.15.169:46214) 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa65400640 1 -- 172.21.15.169:0/1215455094 learned_addr learned my addr 172.21.15.169:0/1215455094 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.375+0000 7ffa65e00640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa680ba460 0x7ffa68153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 -- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6815eb40 msgr2=0x7ffa6815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6815eb40 0x7ffa6815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 -- 172.21.15.169:0/1215455094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa680ba460 msgr2=0x7ffa68153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa680ba460 0x7ffa68153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 -- 172.21.15.169:0/1215455094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa680b6ba0 con 0x7ffa680d1480 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa65400640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa54000c00 tx=0x7ffa540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54032650 con 0x7ffa680d1480 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa540327f0 con 0x7ffa680d1480 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54032be0 con 0x7ffa680d1480 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.376+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa680b5b00 con 0x7ffa680d1480 2024-04-12T19:50:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.377+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa680b60c0 con 0x7ffa680d1480 2024-04-12T19:50:02.378 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.377+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa5404c060 con 0x7ffa680d1480 2024-04-12T19:50:02.378 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.377+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ffa540453f0 con 0x7ffa680d1480 2024-04-12T19:50:02.378 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.378+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa680755c0 con 0x7ffa680d1480 2024-04-12T19:50:02.379 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.380+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffa54040070 con 0x7ffa680d1480 2024-04-12T19:50:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.459+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7661f002-989b-4c53-8297-888c0c4529d2", "id": 7} v 0) -- 0x7ffa680d2360 con 0x7ffa680d1480 2024-04-12T19:50:02.466 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.466+0000 7ffa5e000640 1 -- 172.21.15.169:0/1215455094 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7661f002-989b-4c53-8297-888c0c4529d2", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ffa5404e7e0 con 0x7ffa680d1480 2024-04-12T19:50:02.466 INFO:teuthology.orchestra.run.smithi169.stdout:7 2024-04-12T19:50:02.467 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 msgr2=0x7ffa680b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.467 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa54000c00 tx=0x7ffa540046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.467 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 shutdown_connections 2024-04-12T19:50:02.467 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6815eb40 0x7ffa6815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.467 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa680d1480 0x7ffa680b4b70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.468 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 --2- 172.21.15.169:0/1215455094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa680ba460 0x7ffa68153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.468 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 >> 172.21.15.169:0/1215455094 conn(0x7ffa6806d3d0 msgr2=0x7ffa680c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.468 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 shutdown_connections 2024-04-12T19:50:02.468 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:02.468+0000 7ffa67200640 1 -- 172.21.15.169:0/1215455094 wait complete. 2024-04-12T19:50:02.485 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-12T19:50:02.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-12T19:50:02.492 INFO:tasks.ceph.osd.0:Started 2024-04-12T19:50:02.492 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-12T19:50:02.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-12T19:50:02.501 INFO:tasks.ceph.osd.1:Started 2024-04-12T19:50:02.502 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-12T19:50:02.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-12T19:50:02.506 INFO:tasks.ceph.osd.2:Started 2024-04-12T19:50:02.506 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-12T19:50:02.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-12T19:50:02.511 INFO:tasks.ceph.osd.3:Started 2024-04-12T19:50:02.511 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-12T19:50:02.511 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-12T19:50:02.518 INFO:tasks.ceph.osd.4:Started 2024-04-12T19:50:02.519 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-12T19:50:02.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-12T19:50:02.526 INFO:tasks.ceph.osd.5:Started 2024-04-12T19:50:02.526 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-12T19:50:02.526 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-12T19:50:02.532 INFO:tasks.ceph.osd.6:Started 2024-04-12T19:50:02.532 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-12T19:50:02.532 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-12T19:50:02.538 INFO:tasks.ceph.osd.7:Started 2024-04-12T19:50:02.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:02.559 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:02.560+0000 7fd35f245040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-12T19:50:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.743+0000 7f7bfc000640 1 Processor -- start 2024-04-12T19:50:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.744+0000 7f7bfc000640 1 -- start start 2024-04-12T19:50:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfc000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7bf40cefc0 0x7f7bf40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfc000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 0x7f7bf415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfc000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf40b4be0 con 0x7f7bf41461f0 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfc000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf40a9640 con 0x7f7bf40b3e10 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfc000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf40b4620 con 0x7f7bf40cefc0 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bf9600640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7bf41461f0 0x7f7bf40ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:56836/0 (socket says 172.21.15.32:56836) 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bf9600640 1 -- 172.21.15.32:0/531221627 learned_addr learned my addr 172.21.15.32:0/531221627 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/531221627 >> v2:172.21.15.169:3300/0 conn(0x7f7bf40cefc0 0x7f7bf40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.746+0000 7f7bfa000640 1 --2- 172.21.15.32:0/531221627 >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 0x7f7bf415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf3e00640 1 -- 172.21.15.32:0/531221627 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 449467499 0 0) 0x7f7bf40b4be0 con 0x7f7bf41461f0 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf3e00640 1 -- 172.21.15.32:0/531221627 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bdc003520 con 0x7f7bf41461f0 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf8c00640 1 -- 172.21.15.32:0/531221627 >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 msgr2=0x7f7bf415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/531221627 >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 0x7f7bf415d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf8c00640 1 -- 172.21.15.32:0/531221627 >> v1:172.21.15.169:6789/0 conn(0x7f7bf41461f0 legacy=0x7f7bf40ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bf8c00640 1 -- 172.21.15.32:0/531221627 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf40b4900 con 0x7f7bf40cefc0 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.747+0000 7f7bfa000640 1 --2- 172.21.15.32:0/531221627 >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 0x7f7bf415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.748+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/531221627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf40cefc0 0x7f7bf40b38d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4073560 tx=0x7f7be002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c717f9ced1ceb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.749+0000 7f7bf3e00640 1 -- 172.21.15.32:0/531221627 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0035020 con 0x7f7bf40cefc0 2024-04-12T19:50:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.749+0000 7f7bf3e00640 1 -- 172.21.15.32:0/531221627 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be0031830 con 0x7f7bf40cefc0 2024-04-12T19:50:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.749+0000 7f7bf3e00640 1 -- 172.21.15.32:0/531221627 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0031c00 con 0x7f7bf40cefc0 2024-04-12T19:50:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 -- 172.21.15.32:0/531221627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf40cefc0 msgr2=0x7f7bf40b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 --2- 172.21.15.32:0/531221627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf40cefc0 0x7f7bf40b38d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4073560 tx=0x7f7be002b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 -- 172.21.15.32:0/531221627 shutdown_connections 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 --2- 172.21.15.32:0/531221627 >> v2:172.21.15.32:3300/0 conn(0x7f7bf40b3e10 0x7f7bf415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 --2- 172.21.15.32:0/531221627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf40cefc0 0x7f7bf40b38d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 -- 172.21.15.32:0/531221627 >> 172.21.15.32:0/531221627 conn(0x7f7bf40b27b0 msgr2=0x7f7bf40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 -- 172.21.15.32:0/531221627 shutdown_connections 2024-04-12T19:50:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.750+0000 7f7bfc000640 1 -- 172.21.15.32:0/531221627 wait complete. 2024-04-12T19:50:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.752+0000 7f7bfc000640 1 Processor -- start 2024-04-12T19:50:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.752+0000 7f7bfc000640 1 -- start start 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bf40b3e10 0x7f7bf415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf41461f0 0x7f7bf4140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf41465d0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf415d290 con 0x7f7bf41461f0 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf40b4280 con 0x7f7bf40b3e10 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bf8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bf8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50940/0 (socket says 172.21.15.32:50940) 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bf9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bf40b3e10 0x7f7bf415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bf8c00640 1 -- 172.21.15.32:0/2346497034 learned_addr learned my addr 172.21.15.32:0/2346497034 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.754+0000 7f7bfa000640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf41461f0 0x7f7bf4140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 -- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bf40b3e10 msgr2=0x7f7bf415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bf40b3e10 0x7f7bf415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 -- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf41461f0 msgr2=0x7f7bf4140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf41461f0 0x7f7bf4140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 -- 172.21.15.32:0/2346497034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf41400c0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf8c00640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009890 tx=0x7f7be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0035040 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be003b860 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be003bc50 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf41429b0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.755+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf4142f70 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.756+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7be0038020 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.756+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7be00317b0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.757+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4068c80 con 0x7f7bf40cefc0 2024-04-12T19:50:02.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.759+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7be00457c0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.830+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bf40ce110 con 0x7f7bf40cefc0 2024-04-12T19:50:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.831+0000 7f7bf1600640 1 -- 172.21.15.32:0/2346497034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7be00459a0 con 0x7f7bf40cefc0 2024-04-12T19:50:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:02.832 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:02.461443+0000","last_up_change":"0.000000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.832+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 msgr2=0x7f7bf4143770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.832+0000 7f7bfc000640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009890 tx=0x7f7be00046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.832+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 shutdown_connections 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bf41461f0 0x7f7bf4140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bf40cefc0 0x7f7bf4143770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 --2- 172.21.15.32:0/2346497034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bf40b3e10 0x7f7bf415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 >> 172.21.15.32:0/2346497034 conn(0x7f7bf40b27b0 msgr2=0x7f7bf40afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 shutdown_connections 2024-04-12T19:50:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:02.833+0000 7f7bfc000640 1 -- 172.21.15.32:0/2346497034 wait complete. 2024-04-12T19:50:02.849 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-12T19:50:02.849 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-12T19:50:03.063 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:03.063+0000 7f5085a42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-12T19:50:03.185 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-12T19:50:03.185+0000 7ff85659e640 -1 Falling back to public interface 2024-04-12T19:50:03.203 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-12T19:50:03.204+0000 7ff8c89ff640 -1 Falling back to public interface 2024-04-12T19:50:03.220 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-12T19:50:03.220+0000 7fb9d2419640 -1 Falling back to public interface 2024-04-12T19:50:03.227 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-12T19:50:03.228+0000 7f285e81b640 -1 Falling back to public interface 2024-04-12T19:50:03.228 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-12T19:50:03.228+0000 7fdf8268b640 -1 Falling back to public interface 2024-04-12T19:50:03.236 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-12T19:50:03.236+0000 7fba41d19640 -1 Falling back to public interface 2024-04-12T19:50:03.255 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-12T19:50:03.256+0000 7fc6fb3bb640 -1 Falling back to public interface 2024-04-12T19:50:03.261 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-12T19:50:03.262+0000 7f225b011640 -1 Falling back to public interface 2024-04-12T19:50:03.397 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:03.397+0000 7f5085a42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-12T19:50:03.460 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:03.460+0000 7f5085a42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-12T19:50:03.598 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:03.598+0000 7f5085a42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-12T19:50:03.655 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:03.707 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:03.708+0000 7fd35f245040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-12T19:50:03.806 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.807+0000 7f44d6400640 1 Processor -- start 2024-04-12T19:50:03.806 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.807+0000 7f44d6400640 1 -- start start 2024-04-12T19:50:03.808 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d6400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44d00b4970 0x7f44d0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.808 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d6400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00b5a20 con 0x7f44d00b4590 2024-04-12T19:50:03.808 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d6400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d006f210 con 0x7f44d0159220 2024-04-12T19:50:03.808 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d6400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44d01596b0 con 0x7f44d00b4970 2024-04-12T19:50:03.808 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d5000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44d00b4590 0x7f44d00b4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57772/0 (socket says 172.21.15.32:57772) 2024-04-12T19:50:03.809 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d5000640 1 -- 172.21.15.32:0/1078743978 learned_addr learned my addr 172.21.15.32:0/1078743978 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:03.809 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cfe00640 1 --2- 172.21.15.32:0/1078743978 >> v2:172.21.15.169:3300/0 conn(0x7f44d00b4970 0x7f44d0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.810 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789976566 0 0) 0x7f44d00b5a20 con 0x7f44d00b4590 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44b8003520 con 0x7f44d00b4590 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2788950598 0 0) 0x7f44b8003520 con 0x7f44d00b4590 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 >> v2:172.21.15.169:3300/0 conn(0x7f44d00b4970 msgr2=0x7f44d0158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 --2- 172.21.15.32:0/1078743978 >> v2:172.21.15.169:3300/0 conn(0x7f44d00b4970 0x7f44d0158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 >> v1:172.21.15.169:6789/0 conn(0x7f44d0159220 legacy=0x7f44d006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d006f4f0 con 0x7f44d00b4590 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f44c4001670 con 0x7f44d00b4590 2024-04-12T19:50:03.811 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 >> v1:172.21.15.32:6789/0 conn(0x7f44d00b4590 legacy=0x7f44d00b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b80039e0 0x7f44b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 0x7f44b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b80051c0 0x7f44b800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44d01596b0 con 0x7f44b8004800 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44b800e2a0 con 0x7f44b80051c0 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44b800e4a0 con 0x7f44b80039e0 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44cfe00640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 0x7f44b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.809+0000 7f44d5000640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b80039e0 0x7f44b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44d5a00640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b80051c0 0x7f44b800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44cfe00640 1 -- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b80039e0 msgr2=0x7f44b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44cfe00640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b80039e0 0x7f44b8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.812 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44cfe00640 1 -- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b80051c0 msgr2=0x7f44b800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:03.813 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44cfe00640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b80051c0 0x7f44b800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.813 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44cfe00640 1 -- 172.21.15.32:0/1078743978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b8003520 con 0x7f44b8004800 2024-04-12T19:50:03.813 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.810+0000 7f44d5a00640 1 -- 172.21.15.32:0/1078743978 reap_dead start 2024-04-12T19:50:03.813 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.811+0000 7f44cfe00640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 0x7f44b8004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009ac0 tx=0x7f44b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61cb9c2d75f435c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:03.814 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.811+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc039070 con 0x7f44b8004800 2024-04-12T19:50:03.814 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.811+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc02c6b0 con 0x7f44b8004800 2024-04-12T19:50:03.814 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.811+0000 7f44cf400640 1 -- 172.21.15.32:0/1078743978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009690 con 0x7f44b8004800 2024-04-12T19:50:03.814 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 -- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 msgr2=0x7f44b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 0x7f44b8004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009ac0 tx=0x7f44b8005830 comp rx=0 tx=0).stop 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 -- 172.21.15.32:0/1078743978 shutdown_connections 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 --2- 172.21.15.32:0/1078743978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b8004800 0x7f44b8004c80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 -- 172.21.15.32:0/1078743978 >> 172.21.15.32:0/1078743978 conn(0x7f44d009aa30 msgr2=0x7f44d00b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 -- 172.21.15.32:0/1078743978 shutdown_connections 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.812+0000 7f44d6400640 1 -- 172.21.15.32:0/1078743978 wait complete. 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 Processor -- start 2024-04-12T19:50:03.815 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 -- start start 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 0x7f44d009ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d014cf10 0x7f44d009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44d015cb20 con 0x7f44d015a090 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44d006efc0 con 0x7f44b8004800 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.815+0000 7f44d6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44d0158ce0 con 0x7f44d014cf10 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51030/0 (socket says 172.21.15.32:51030) 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 -- 172.21.15.32:0/3856506692 learned_addr learned my addr 172.21.15.32:0/3856506692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:03.817 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5000640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 0x7f44d009ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44cfe00640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d014cf10 0x7f44d009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 -- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d014cf10 msgr2=0x7f44d009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d014cf10 0x7f44d009c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 -- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 msgr2=0x7f44d009ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 0x7f44d009ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 -- 172.21.15.32:0/3856506692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d01598d0 con 0x7f44d015a090 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5000640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 0x7f44d009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.816+0000 7f44d5a00640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44c8007fd0 tx=0x7f44c800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.817+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8019070 con 0x7f44d015a090 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.817+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c8006d60 con 0x7f44d015a090 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.817+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8003b90 con 0x7f44d015a090 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.817+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d009d250 con 0x7f44d015a090 2024-04-12T19:50:03.818 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.817+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d0159b70 con 0x7f44d015a090 2024-04-12T19:50:03.819 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.818+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 2) ==== 45545+0+0 (secure 0 0 0) 0x7f44c801c070 con 0x7f44d015a090 2024-04-12T19:50:03.819 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.818+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d009cfe0 con 0x7f44d015a090 2024-04-12T19:50:03.819 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.818+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f44c804a000 con 0x7f44d015a090 2024-04-12T19:50:03.820 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.821+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f44c8034e50 con 0x7f44d015a090 2024-04-12T19:50:03.889 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.889+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44d004fd50 con 0x7f44d015a090 2024-04-12T19:50:03.891 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.891+0000 7f44ccc00640 1 -- 172.21.15.32:0/3856506692 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f44c8005070 con 0x7f44d015a090 2024-04-12T19:50:03.891 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-12T19:50:03.891 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:02.461443+0000","last_up_change":"0.000000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:03.892 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 msgr2=0x7f44d00cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:03.892 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44c8007fd0 tx=0x7f44c800ea00 comp rx=0 tx=0).stop 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 shutdown_connections 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44d015a090 0x7f44d00cb730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d014cf10 0x7f44d009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 --2- 172.21.15.32:0/3856506692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b8004800 0x7f44d009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 >> 172.21.15.32:0/3856506692 conn(0x7f44d009aa30 msgr2=0x7f44d00aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 shutdown_connections 2024-04-12T19:50:03.893 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:03.893+0000 7f44d6400640 1 -- 172.21.15.32:0/3856506692 wait complete. 2024-04-12T19:50:03.906 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-12T19:50:04.028 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:04.028+0000 7fd35f245040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-12T19:50:04.091 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:04.092+0000 7fd35f245040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-12T19:50:04.224 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-12T19:50:04.225+0000 7fd35f245040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-12T19:50:04.635 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:04.636+0000 7f5026000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-12T19:50:04.639 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-12T19:50:04.640+0000 7f5026000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-12T19:50:06.635 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-12T19:50:06.636+0000 7f225b011640 -1 osd.7 0 log_to_monitors true 2024-04-12T19:50:06.800 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-12T19:50:06.801+0000 7ff8c89ff640 -1 osd.4 0 log_to_monitors true 2024-04-12T19:50:06.830 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-12T19:50:06.831+0000 7ff85659e640 -1 osd.1 0 log_to_monitors true 2024-04-12T19:50:06.862 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-12T19:50:06.863+0000 7f285e81b640 -1 osd.6 0 log_to_monitors true 2024-04-12T19:50:06.867 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-12T19:50:06.868+0000 7fb9d2419640 -1 osd.3 0 log_to_monitors true 2024-04-12T19:50:06.868 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-12T19:50:06.869+0000 7fba41d19640 -1 osd.2 0 log_to_monitors true 2024-04-12T19:50:06.876 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-12T19:50:06.877+0000 7fdf8268b640 -1 osd.0 0 log_to_monitors true 2024-04-12T19:50:06.912 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-12T19:50:06.913+0000 7fc6fb3bb640 -1 osd.5 0 log_to_monitors true 2024-04-12T19:50:09.196 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-12T19:50:09.196+0000 7f2255e00640 -1 osd.7 0 waiting for initial osdmap 2024-04-12T19:50:09.366 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-12T19:50:09.366+0000 7f224b400640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:09.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:50:09.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:50:10.428 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-12T19:50:10.428+0000 7fdf7d200640 -1 osd.0 0 waiting for initial osdmap 2024-04-12T19:50:10.433 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-12T19:50:10.434+0000 7fba3d200640 -1 osd.2 0 waiting for initial osdmap 2024-04-12T19:50:10.434 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-12T19:50:10.435+0000 7fdf73200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.443 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-12T19:50:10.444+0000 7fba32800640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.447 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-12T19:50:10.448+0000 7ff851000640 -1 osd.1 0 waiting for initial osdmap 2024-04-12T19:50:10.449 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-12T19:50:10.450+0000 7fb9cca00640 -1 osd.3 0 waiting for initial osdmap 2024-04-12T19:50:10.457 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-12T19:50:10.458+0000 7ff847000640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.465 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-12T19:50:10.466+0000 7fb9c2a00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.598 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-12T19:50:10.598+0000 7f2858c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-12T19:50:10.609 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-12T19:50:10.607+0000 7f284ec00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.619 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-12T19:50:10.619+0000 7ff8c3400640 -1 osd.4 0 waiting for initial osdmap 2024-04-12T19:50:10.628 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-12T19:50:10.627+0000 7ff8b9400640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.662 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-12T19:50:10.662+0000 7fc6f6800640 -1 osd.5 0 waiting for initial osdmap 2024-04-12T19:50:10.668 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-12T19:50:10.668+0000 7fc6ebe00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T19:50:10.713 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:10.886 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.886+0000 7fcbc3e00640 1 Processor -- start 2024-04-12T19:50:10.886 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.887+0000 7fcbc3e00640 1 -- start start 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc415ec20 0x7fcbc40d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc40d3660 con 0x7fcbc40cff80 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc40d3940 con 0x7fcbc40b7050 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc40d3f90 con 0x7fcbc415ec20 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc415ec20 0x7fcbc40d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcbc40cff80 0x7fcbc40b7c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36110/0 (socket says 172.21.15.32:36110) 2024-04-12T19:50:10.887 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc3400640 1 -- 172.21.15.32:0/1739428808 learned_addr learned my addr 172.21.15.32:0/1739428808 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:10.888 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc2a00640 1 -- 172.21.15.32:0/1739428808 >> v1:172.21.15.169:6789/0 conn(0x7fcbc40b7050 legacy=0x7fcbc40cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:10.888 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc2a00640 1 -- 172.21.15.32:0/1739428808 >> v1:172.21.15.32:6789/0 conn(0x7fcbc40cff80 legacy=0x7fcbc40b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:10.888 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc2a00640 1 -- 172.21.15.32:0/1739428808 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc40d3c20 con 0x7fcbc415ec20 2024-04-12T19:50:10.888 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.888+0000 7fcbc2000640 1 -- 172.21.15.32:0/1739428808 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948136301 0 0) 0x7fcbc40d3660 con 0x7fcbc40cff80 2024-04-12T19:50:10.888 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc2a00640 1 --2- 172.21.15.32:0/1739428808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc415ec20 0x7fcbc40d19c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40097a0 tx=0x7fcbb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d2ad719fa9890b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:10.889 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc2000640 1 -- 172.21.15.32:0/1739428808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4035020 con 0x7fcbc415ec20 2024-04-12T19:50:10.889 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc2000640 1 -- 172.21.15.32:0/1739428808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb4031640 con 0x7fcbc415ec20 2024-04-12T19:50:10.889 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc2000640 1 -- 172.21.15.32:0/1739428808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb402d040 con 0x7fcbc415ec20 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc3e00640 1 -- 172.21.15.32:0/1739428808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc415ec20 msgr2=0x7fcbc40d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.890+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/1739428808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc415ec20 0x7fcbc40d19c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40097a0 tx=0x7fcbb402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.891+0000 7fcbc3e00640 1 -- 172.21.15.32:0/1739428808 shutdown_connections 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.891+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/1739428808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc415ec20 0x7fcbc40d19c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.891+0000 7fcbc3e00640 1 -- 172.21.15.32:0/1739428808 >> 172.21.15.32:0/1739428808 conn(0x7fcbc40736a0 msgr2=0x7fcbc40d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.891+0000 7fcbc3e00640 1 -- 172.21.15.32:0/1739428808 shutdown_connections 2024-04-12T19:50:10.890 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.891+0000 7fcbc3e00640 1 -- 172.21.15.32:0/1739428808 wait complete. 2024-04-12T19:50:10.892 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.893+0000 7fcbc3e00640 1 Processor -- start 2024-04-12T19:50:10.892 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.893+0000 7fcbc3e00640 1 -- start start 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 0x7fcbc40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc415ec20 0x7fcbc4146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc40d3410 con 0x7fcbc40cff80 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc40d2110 con 0x7fcbc415ec20 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc40b8350 con 0x7fcbc40b7050 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:10.895 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42734/0 (socket says 172.21.15.32:42734) 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 -- 172.21.15.32:0/795708540 learned_addr learned my addr 172.21.15.32:0/795708540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc2a00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 0x7fcbc40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc3400640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc415ec20 0x7fcbc4146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 -- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 msgr2=0x7fcbc40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 0x7fcbc40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 -- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc415ec20 msgr2=0x7fcbc4146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc415ec20 0x7fcbc4146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbba000640 1 -- 172.21.15.32:0/795708540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc41f8e20 con 0x7fcbc40cff80 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.894+0000 7fcbc2a00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 0x7fcbc40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbba000640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c430 tx=0x7fcbac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac011790 con 0x7fcbc40cff80 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac017070 con 0x7fcbc40cff80 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac011bb0 con 0x7fcbc40cff80 2024-04-12T19:50:10.896 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc40cdf70 con 0x7fcbc40cff80 2024-04-12T19:50:10.897 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.895+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc40ce530 con 0x7fcbc40cff80 2024-04-12T19:50:10.897 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.896+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc404fd50 con 0x7fcbc40cff80 2024-04-12T19:50:10.897 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.897+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fcbac01c460 con 0x7fcbc40cff80 2024-04-12T19:50:10.897 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.898+0000 7fcbbb400640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 0x7fcba006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:10.897 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.898+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7fcbac08ad00 con 0x7fcbc40cff80 2024-04-12T19:50:10.898 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.898+0000 7fcbc2a00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 0x7fcba006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:10.898 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.899+0000 7fcbc2a00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 0x7fcba006ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4036000 tx=0x7fcbb403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:10.902 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:10.903+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbc41f8e20 con 0x7fcbc40cff80 2024-04-12T19:50:11.100 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.100+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbc40d1be0 con 0x7fcbc40cff80 2024-04-12T19:50:11.101 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.102+0000 7fcbbb400640 1 -- 172.21.15.32:0/795708540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7555 (secure 0 0 0) 0x7fcbac059df0 con 0x7fcbc40cff80 2024-04-12T19:50:11.101 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-12T19:50:11.101 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":14,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:10.210469+0000","last_up_change":"2024-04-12T19:50:10.210469+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.104+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 msgr2=0x7fcba006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.104+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 0x7fcba006ed50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4036000 tx=0x7fcbb403c040 comp rx=0 tx=0).stop 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.104+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 msgr2=0x7fcbc41f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.104+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c430 tx=0x7fcbac00c8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 shutdown_connections 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcba006c890 0x7fcba006ed50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc415ec20 0x7fcbc4146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:11.104 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc40cff80 0x7fcbc41f88e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:11.105 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 --2- 172.21.15.32:0/795708540 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc40b7050 0x7fcbc40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:11.105 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 >> 172.21.15.32:0/795708540 conn(0x7fcbc40736a0 msgr2=0x7fcbc415cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:11.105 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 shutdown_connections 2024-04-12T19:50:11.105 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:11.105+0000 7fcbc3e00640 1 -- 172.21.15.32:0/795708540 wait complete. 2024-04-12T19:50:11.117 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-12T19:50:17.925 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:18.124 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.124+0000 7f1f28a00640 1 Processor -- start 2024-04-12T19:50:18.125 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.124+0000 7f1f28a00640 1 -- start start 2024-04-12T19:50:18.126 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.126+0000 7f1f28a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f2415f890 0x7f1f24072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.126 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.126+0000 7f1f28a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f240b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.126 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.126+0000 7f1f28a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f24072c80 con 0x7f1f2415f4b0 2024-04-12T19:50:18.126 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.126+0000 7f1f28a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1f241630b0 con 0x7f1f2415f890 2024-04-12T19:50:18.127 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.126+0000 7f1f28a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2415fd40 con 0x7f1f24072880 2024-04-12T19:50:18.127 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f2415f890 0x7f1f24072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f2415f890 0x7f1f24072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55102/0 (socket says 172.21.15.32:55102) 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 -- 172.21.15.32:0/1761177487 learned_addr learned my addr 172.21.15.32:0/1761177487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f23e00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f240b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 -- 172.21.15.32:0/1761177487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 msgr2=0x7f1f240b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f240b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 -- 172.21.15.32:0/1761177487 >> v1:172.21.15.169:6789/0 conn(0x7f1f2415f4b0 legacy=0x7f1f240b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.128 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.127+0000 7f1f22a00640 1 -- 172.21.15.32:0/1761177487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f241624a0 con 0x7f1f2415f890 2024-04-12T19:50:18.129 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.129+0000 7f1f22a00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f24072340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009870 tx=0x7f1f1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d877f047f2bbd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.129 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.129+0000 7f1f22000640 1 -- 172.21.15.32:0/1761177487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10034070 con 0x7f1f2415f890 2024-04-12T19:50:18.129 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.129+0000 7f1f22000640 1 -- 172.21.15.32:0/1761177487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10030840 con 0x7f1f2415f890 2024-04-12T19:50:18.130 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.129+0000 7f1f22000640 1 -- 172.21.15.32:0/1761177487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10030c10 con 0x7f1f2415f890 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 -- 172.21.15.32:0/1761177487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 msgr2=0x7f1f24072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f24072340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009870 tx=0x7f1f1002b980 comp rx=0 tx=0).stop 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 -- 172.21.15.32:0/1761177487 shutdown_connections 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f240b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 --2- 172.21.15.32:0/1761177487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f24072340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 -- 172.21.15.32:0/1761177487 >> 172.21.15.32:0/1761177487 conn(0x7f1f2406bd30 msgr2=0x7f1f2406c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 -- 172.21.15.32:0/1761177487 shutdown_connections 2024-04-12T19:50:18.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.130+0000 7f1f28a00640 1 -- 172.21.15.32:0/1761177487 wait complete. 2024-04-12T19:50:18.132 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.132+0000 7f1f28a00640 1 Processor -- start 2024-04-12T19:50:18.132 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.132+0000 7f1f28a00640 1 -- start start 2024-04-12T19:50:18.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f24156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2415f4b0 0x7f1f241f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24162d40 con 0x7f1f2415f890 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24162a60 con 0x7f1f2415f4b0 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.133+0000 7f1f28a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f240b3da0 con 0x7f1f24072880 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55114/0 (socket says 172.21.15.32:55114) 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 -- 172.21.15.32:0/468230230 learned_addr learned my addr 172.21.15.32:0/468230230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f22a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2415f4b0 0x7f1f241f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23400640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f24156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 -- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 msgr2=0x7f1f24156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f24156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 -- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2415f4b0 msgr2=0x7f1f241f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2415f4b0 0x7f1f241f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.134+0000 7f1f23e00640 1 -- 172.21.15.32:0/468230230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f241eab60 con 0x7f1f2415f890 2024-04-12T19:50:18.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f23e00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00efc0 tx=0x7f1f1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c018020 con 0x7f1f2415f890 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c014480 con 0x7f1f2415f890 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c014870 con 0x7f1f2415f890 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f24157100 con 0x7f1f2415f890 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.135+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2415a110 con 0x7f1f2415f890 2024-04-12T19:50:18.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.136+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2404fd50 con 0x7f1f2415f890 2024-04-12T19:50:18.137 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.137+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f1c005d10 con 0x7f1f2415f890 2024-04-12T19:50:18.138 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.138+0000 7f1f1b400640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.138 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.138+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1f1c08c650 con 0x7f1f2415f890 2024-04-12T19:50:18.138 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.138+0000 7f1f23400640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.139+0000 7f1f23400640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 0x7f1f0406edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001640 tx=0x7f1f14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.144 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.144+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f1c022030 con 0x7f1f2415f890 2024-04-12T19:50:18.359 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.359+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f2406cf80 con 0x7f1f2415f890 2024-04-12T19:50:18.361 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.361+0000 7f1f1b400640 1 -- 172.21.15.32:0/468230230 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12872 (secure 0 0 0) 0x7f1f1c010040 con 0x7f1f2415f890 2024-04-12T19:50:18.361 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-12T19:50:18.362 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":18,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:14.442467+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:18.364 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.363+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 msgr2=0x7f1f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.364 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.363+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 0x7f1f0406edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001640 tx=0x7f1f14009040 comp rx=0 tx=0).stop 2024-04-12T19:50:18.364 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 msgr2=0x7f1f241ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.364 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00efc0 tx=0x7f1f1c00c490 comp rx=0 tx=0).stop 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 shutdown_connections 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f0406c8f0 0x7f1f0406edb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f2415f890 0x7f1f241ea100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2415f4b0 0x7f1f241f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 --2- 172.21.15.32:0/468230230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f24072880 0x7f1f24156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 >> 172.21.15.32:0/468230230 conn(0x7f1f2406bd30 msgr2=0x7f1f240b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 shutdown_connections 2024-04-12T19:50:18.365 INFO:teuthology.misc.health.smithi032.stderr:2024-04-12T19:50:18.364+0000 7f1f28a00640 1 -- 172.21.15.32:0/468230230 wait complete. 2024-04-12T19:50:18.379 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-12T19:50:18.379 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-12T19:50:18.380 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-12T19:50:18.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-12T19:50:18.387 INFO:tasks.ceph.mds.a:Started 2024-04-12T19:50:18.387 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-12T19:50:18.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-12T19:50:18.394 INFO:tasks.ceph.mds.c:Started 2024-04-12T19:50:18.394 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-12T19:50:18.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-12T19:50:18.406 INFO:tasks.ceph.mds.b:Started 2024-04-12T19:50:18.406 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-12T19:50:18.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-12T19:50:18.413 INFO:tasks.ceph.mds.d:Started 2024-04-12T19:50:18.413 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-12T19:50:18.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:18.546 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-12T19:50:18.547 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-12T19:50:18.575 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-12T19:50:18.576 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-12T19:50:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.606+0000 7f9c59600640 1 Processor -- start 2024-04-12T19:50:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.606+0000 7f9c59600640 1 -- start start 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c59600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 0x7f9c540d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c59600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 0x7f9c540b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c59600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c540b41d0 con 0x7f9c54162970 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c59600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c54162780 con 0x7f9c540b63e0 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c59600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c540b6d80 con 0x7f9c540b3df0 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c53400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9c54162970 0x7f9c54162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51128/0 (socket says 172.21.15.32:51128) 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c53400640 1 -- 172.21.15.32:0/916982328 learned_addr learned my addr 172.21.15.32:0/916982328 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c52a00640 1 --2- 172.21.15.32:0/916982328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 0x7f9c540d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.607+0000 7f9c4be00640 1 --2- 172.21.15.32:0/916982328 >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 0x7f9c540b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52000640 1 -- 172.21.15.32:0/916982328 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730226408 0 0) 0x7f9c540b41d0 con 0x7f9c54162970 2024-04-12T19:50:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52000640 1 -- 172.21.15.32:0/916982328 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c38003520 con 0x7f9c54162970 2024-04-12T19:50:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52a00640 1 -- 172.21.15.32:0/916982328 >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 msgr2=0x7f9c540b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52a00640 1 --2- 172.21.15.32:0/916982328 >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 0x7f9c540b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52a00640 1 -- 172.21.15.32:0/916982328 >> v1:172.21.15.32:6789/0 conn(0x7f9c54162970 legacy=0x7f9c54162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c52a00640 1 -- 172.21.15.32:0/916982328 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c540cfa40 con 0x7f9c540b3df0 2024-04-12T19:50:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.608+0000 7f9c4be00640 1 --2- 172.21.15.32:0/916982328 >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 0x7f9c540b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.609+0000 7f9c52a00640 1 --2- 172.21.15.32:0/916982328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 0x7f9c540d4330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c440097a0 tx=0x7f9c4402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4669eb608fe63e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.609+0000 7f9c52000640 1 -- 172.21.15.32:0/916982328 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44035020 con 0x7f9c540b3df0 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.609+0000 7f9c52000640 1 -- 172.21.15.32:0/916982328 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c44031830 con 0x7f9c540b3df0 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.609+0000 7f9c52000640 1 -- 172.21.15.32:0/916982328 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44031c00 con 0x7f9c540b3df0 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 -- 172.21.15.32:0/916982328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 msgr2=0x7f9c540d4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 --2- 172.21.15.32:0/916982328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 0x7f9c540d4330 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c440097a0 tx=0x7f9c4402b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 -- 172.21.15.32:0/916982328 shutdown_connections 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 --2- 172.21.15.32:0/916982328 >> v2:172.21.15.169:3300/0 conn(0x7f9c540b63e0 0x7f9c540b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 --2- 172.21.15.32:0/916982328 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b3df0 0x7f9c540d4330 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 -- 172.21.15.32:0/916982328 >> 172.21.15.32:0/916982328 conn(0x7f9c5406d5f0 msgr2=0x7f9c540d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 -- 172.21.15.32:0/916982328 shutdown_connections 2024-04-12T19:50:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.610+0000 7f9c59600640 1 -- 172.21.15.32:0/916982328 wait complete. 2024-04-12T19:50:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.612+0000 7f9c59600640 1 Processor -- start 2024-04-12T19:50:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.612+0000 7f9c59600640 1 -- start start 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540b3df0 0x7f9c541f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 0x7f9c540cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c540d22b0 con 0x7f9c540b63e0 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c540d2fe0 con 0x7f9c540b3df0 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c59600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c540b8aa0 con 0x7f9c54162970 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55152/0 (socket says 172.21.15.32:55152) 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.613+0000 7f9c4be00640 1 -- 172.21.15.32:0/4046677377 learned_addr learned my addr 172.21.15.32:0/4046677377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c53400640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 0x7f9c540cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c52a00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540b3df0 0x7f9c541f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 -- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 msgr2=0x7f9c540cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 0x7f9c540cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 -- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540b3df0 msgr2=0x7f9c541f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540b3df0 0x7f9c541f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 -- 172.21.15.32:0/4046677377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c540ad050 con 0x7f9c540b63e0 2024-04-12T19:50:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c53400640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 0x7f9c540cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4be00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00ec10 tx=0x7f9c3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.614+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c017070 con 0x7f9c540b63e0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.615+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00cc30 con 0x7f9c540b63e0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.615+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c540ad330 con 0x7f9c540b63e0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.615+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c013470 con 0x7f9c540b63e0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.615+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c540ad8f0 con 0x7f9c540b63e0 2024-04-12T19:50:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.616+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c540d0540 con 0x7f9c540b63e0 2024-04-12T19:50:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.617+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c3c01a070 con 0x7f9c540b63e0 2024-04-12T19:50:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.618+0000 7f9c4aa00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 0x7f9c3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.618+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f9c3c08bb00 con 0x7f9c540b63e0 2024-04-12T19:50:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.618+0000 7f9c52a00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 0x7f9c3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:18.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.618+0000 7f9c52a00640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 0x7f9c3006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c44036000 tx=0x7f9c4403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.622+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c540d0540 con 0x7f9c540b63e0 2024-04-12T19:50:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.829+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c54068c80 con 0x7f9c540b63e0 2024-04-12T19:50:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.830+0000 7f9c4aa00640 1 -- 172.21.15.32:0/4046677377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12872 (secure 0 0 0) 0x7f9c3c059b90 con 0x7f9c540b63e0 2024-04-12T19:50:18.831 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:18.831 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:14.442467+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 msgr2=0x7f9c3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 0x7f9c3006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c44036000 tx=0x7f9c4403d040 comp rx=0 tx=0).stop 2024-04-12T19:50:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 msgr2=0x7f9c540ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00ec10 tx=0x7f9c3c00c520 comp rx=0 tx=0).stop 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 shutdown_connections 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c3006c8f0 0x7f9c3006edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c54162970 0x7f9c540cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c540b63e0 0x7f9c540ade10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 --2- 172.21.15.32:0/4046677377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540b3df0 0x7f9c541f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 >> 172.21.15.32:0/4046677377 conn(0x7f9c5406d5f0 msgr2=0x7f9c54161a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 shutdown_connections 2024-04-12T19:50:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:18.834+0000 7f9c59600640 1 -- 172.21.15.32:0/4046677377 wait complete. 2024-04-12T19:50:18.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:18.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.071+0000 7fd53ce00640 1 Processor -- start 2024-04-12T19:50:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.072+0000 7fd53ce00640 1 -- start start 2024-04-12T19:50:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.073+0000 7fd53ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5380d26b0 0x7fd5380b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.073+0000 7fd53ce00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5380cfef0 con 0x7fd5380b7e80 2024-04-12T19:50:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.073+0000 7fd53ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5380d2bb0 con 0x7fd538161f80 2024-04-12T19:50:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.073+0000 7fd53ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd5380d2e90 con 0x7fd5380d26b0 2024-04-12T19:50:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.074+0000 7fd537e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5380b7e80 0x7fd5380d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51140/0 (socket says 172.21.15.32:51140) 2024-04-12T19:50:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.074+0000 7fd537e00640 1 -- 172.21.15.32:0/525222878 learned_addr learned my addr 172.21.15.32:0/525222878 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.074+0000 7fd536a00640 1 --2- 172.21.15.32:0/525222878 >> v2:172.21.15.169:3300/0 conn(0x7fd5380d26b0 0x7fd5380b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.074+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399276439 0 0) 0x7fd5380cfef0 con 0x7fd5380b7e80 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.074+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd51c003520 con 0x7fd5380b7e80 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2445164313 0 0) 0x7fd51c003520 con 0x7fd5380b7e80 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 >> v2:172.21.15.169:3300/0 conn(0x7fd5380d26b0 msgr2=0x7fd5380b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 --2- 172.21.15.32:0/525222878 >> v2:172.21.15.169:3300/0 conn(0x7fd5380d26b0 0x7fd5380b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 >> v1:172.21.15.169:6789/0 conn(0x7fd538161f80 legacy=0x7fd5380cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5380d3170 con 0x7fd5380b7e80 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fd5300019f0 con 0x7fd5380b7e80 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 >> v1:172.21.15.32:6789/0 conn(0x7fd5380b7e80 legacy=0x7fd5380d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0039b0 0x7fd51c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c004800 0x7fd51c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 0x7fd51c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5380d2e90 con 0x7fd51c0051c0 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c00c040 con 0x7fd51c0039b0 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c00c240 con 0x7fd51c004800 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd537e00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 0x7fd51c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd537400640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0039b0 0x7fd51c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd537e00640 1 -- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c004800 msgr2=0x7fd51c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.075+0000 7fd537e00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c004800 0x7fd51c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.076+0000 7fd537e00640 1 -- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0039b0 msgr2=0x7fd51c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.076+0000 7fd537e00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0039b0 0x7fd51c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.076+0000 7fd537e00640 1 -- 172.21.15.32:0/525222878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c003520 con 0x7fd51c0051c0 2024-04-12T19:50:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.076+0000 7fd537e00640 1 -- 172.21.15.32:0/525222878 reap_dead start 2024-04-12T19:50:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd537e00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 0x7fd51c00b9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd53000a920 tx=0x7fd51c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69a1e0945eea5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53000a4c0 con 0x7fd51c0051c0 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530034650 con 0x7fd51c0051c0 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd536000640 1 -- 172.21.15.32:0/525222878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530034a40 con 0x7fd51c0051c0 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd53ce00640 1 -- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 msgr2=0x7fd51c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.077+0000 7fd53ce00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 0x7fd51c00b9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd53000a920 tx=0x7fd51c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.078+0000 7fd53ce00640 1 -- 172.21.15.32:0/525222878 shutdown_connections 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.078+0000 7fd53ce00640 1 --2- 172.21.15.32:0/525222878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd51c0051c0 0x7fd51c00b9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.078+0000 7fd53ce00640 1 -- 172.21.15.32:0/525222878 >> 172.21.15.32:0/525222878 conn(0x7fd53806ce40 msgr2=0x7fd53806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.078+0000 7fd53ce00640 1 -- 172.21.15.32:0/525222878 shutdown_connections 2024-04-12T19:50:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.078+0000 7fd53ce00640 1 -- 172.21.15.32:0/525222878 wait complete. 2024-04-12T19:50:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.079+0000 7fd53ce00640 1 Processor -- start 2024-04-12T19:50:19.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 -- start start 2024-04-12T19:50:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0051c0 0x7fd5380c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381f7780 0x7fd538157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5380b7940 con 0x7fd5381f7bc0 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd538162360 con 0x7fd51c0051c0 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.080+0000 7fd53ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5380b7ac0 con 0x7fd5381f7780 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55172/0 (socket says 172.21.15.32:55172) 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 -- 172.21.15.32:0/1430557774 learned_addr learned my addr 172.21.15.32:0/1430557774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd536a00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381f7780 0x7fd538157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537400640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0051c0 0x7fd5380c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 -- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381f7780 msgr2=0x7fd538157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381f7780 0x7fd538157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 -- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0051c0 msgr2=0x7fd5380c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0051c0 0x7fd5380c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.081+0000 7fd537e00640 1 -- 172.21.15.32:0/1430557774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd538158b00 con 0x7fd5381f7bc0 2024-04-12T19:50:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd537e00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd53002cc90 tx=0x7fd530001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530043070 con 0x7fd5381f7bc0 2024-04-12T19:50:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5300019b0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530039040 con 0x7fd5381f7bc0 2024-04-12T19:50:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd538158820 con 0x7fd5381f7bc0 2024-04-12T19:50:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.082+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5380c6ef0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.083+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5380755c0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.084+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd530046070 con 0x7fd5381f7bc0 2024-04-12T19:50:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.086+0000 7fd52f400640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 0x7fd51806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.086+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd5300acbd0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.086+0000 7fd537400640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 0x7fd51806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.087+0000 7fd537400640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 0x7fd51806ede0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd528004dc0 tx=0x7fd528009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.091+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd53007b2c0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.291+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5380d2490 con 0x7fd5381f7bc0 2024-04-12T19:50:19.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.292+0000 7fd52f400640 1 -- 172.21.15.32:0/1430557774 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7fd53007abc0 con 0x7fd5381f7bc0 2024-04-12T19:50:19.292 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.294+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 msgr2=0x7fd51806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.294+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 0x7fd51806ede0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd528004dc0 tx=0x7fd528009040 comp rx=0 tx=0).stop 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.294+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 msgr2=0x7fd538159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.294+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd53002cc90 tx=0x7fd530001dc0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 shutdown_connections 2024-04-12T19:50:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd51806c920 0x7fd51806ede0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5381f7bc0 0x7fd538159f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381f7780 0x7fd538157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 --2- 172.21.15.32:0/1430557774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c0051c0 0x7fd5380c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 >> 172.21.15.32:0/1430557774 conn(0x7fd53806ce40 msgr2=0x7fd538145900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 shutdown_connections 2024-04-12T19:50:19.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.295+0000 7fd53ce00640 1 -- 172.21.15.32:0/1430557774 wait complete. 2024-04-12T19:50:19.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:50:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.515+0000 7fe767e00640 1 Processor -- start 2024-04-12T19:50:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.515+0000 7fe767e00640 1 -- start start 2024-04-12T19:50:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.516+0000 7fe767e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7680b3c70 0x7fe7680d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe767e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76815ef90 con 0x7fe7680d1af0 2024-04-12T19:50:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe767e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7680cf090 con 0x7fe76815ebb0 2024-04-12T19:50:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe767e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe7680d1f20 con 0x7fe7680b3c70 2024-04-12T19:50:19.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe767400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7680d1af0 0x7fe7680d2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51150/0 (socket says 172.21.15.32:51150) 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe767400640 1 -- 172.21.15.32:0/2558218907 learned_addr learned my addr 172.21.15.32:0/2558218907 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe766000640 1 --2- 172.21.15.32:0/2558218907 >> v2:172.21.15.169:3300/0 conn(0x7fe7680b3c70 0x7fe7680d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe765600640 1 -- 172.21.15.32:0/2558218907 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378756429 0 0) 0x7fe76815ef90 con 0x7fe7680d1af0 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.517+0000 7fe765600640 1 -- 172.21.15.32:0/2558218907 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe748003520 con 0x7fe7680d1af0 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.518+0000 7fe766000640 1 -- 172.21.15.32:0/2558218907 >> v1:172.21.15.169:6789/0 conn(0x7fe76815ebb0 legacy=0x7fe768163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.518+0000 7fe766000640 1 -- 172.21.15.32:0/2558218907 >> v1:172.21.15.32:6789/0 conn(0x7fe7680d1af0 legacy=0x7fe7680d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.518+0000 7fe766000640 1 -- 172.21.15.32:0/2558218907 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680cf370 con 0x7fe7680b3c70 2024-04-12T19:50:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.519+0000 7fe766000640 1 --2- 172.21.15.32:0/2558218907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680b3c70 0x7fe7680d15b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d90 tx=0x7fe75002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a2a909bd85b535 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.520+0000 7fe765600640 1 -- 172.21.15.32:0/2558218907 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750039070 con 0x7fe7680b3c70 2024-04-12T19:50:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.520+0000 7fe765600640 1 -- 172.21.15.32:0/2558218907 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75002cd00 con 0x7fe7680b3c70 2024-04-12T19:50:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.520+0000 7fe765600640 1 -- 172.21.15.32:0/2558218907 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750034660 con 0x7fe7680b3c70 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.520+0000 7fe767e00640 1 -- 172.21.15.32:0/2558218907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680b3c70 msgr2=0x7fe7680d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.520+0000 7fe767e00640 1 --2- 172.21.15.32:0/2558218907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680b3c70 0x7fe7680d15b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe750009d90 tx=0x7fe75002bf50 comp rx=0 tx=0).stop 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.521+0000 7fe767e00640 1 -- 172.21.15.32:0/2558218907 shutdown_connections 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.521+0000 7fe767e00640 1 --2- 172.21.15.32:0/2558218907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680b3c70 0x7fe7680d15b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.521+0000 7fe767e00640 1 -- 172.21.15.32:0/2558218907 >> 172.21.15.32:0/2558218907 conn(0x7fe76806d040 msgr2=0x7fe76806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.521+0000 7fe767e00640 1 -- 172.21.15.32:0/2558218907 shutdown_connections 2024-04-12T19:50:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.521+0000 7fe767e00640 1 -- 172.21.15.32:0/2558218907 wait complete. 2024-04-12T19:50:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.523+0000 7fe767e00640 1 Processor -- start 2024-04-12T19:50:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.523+0000 7fe767e00640 1 -- start start 2024-04-12T19:50:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680d1af0 0x7fe7680b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76815ebb0 0x7fe76815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680cfc10 con 0x7fe7680cb900 2024-04-12T19:50:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680cf930 con 0x7fe7680d1af0 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680cfef0 con 0x7fe76815ebb0 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55174/0 (socket says 172.21.15.32:55174) 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe767400640 1 -- 172.21.15.32:0/1688911045 learned_addr learned my addr 172.21.15.32:0/1688911045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe766000640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76815ebb0 0x7fe76815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.525+0000 7fe766a00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680d1af0 0x7fe7680b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 -- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76815ebb0 msgr2=0x7fe76815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76815ebb0 0x7fe76815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 -- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680d1af0 msgr2=0x7fe7680b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680d1af0 0x7fe7680b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 -- 172.21.15.32:0/1688911045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768160440 con 0x7fe7680cb900 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767400640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c430 tx=0x7fe76000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760017070 con 0x7fe7680cb900 2024-04-12T19:50:19.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760005ce0 con 0x7fe7680cb900 2024-04-12T19:50:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760007310 con 0x7fe7680cb900 2024-04-12T19:50:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.526+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7680cc910 con 0x7fe7680cb900 2024-04-12T19:50:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.527+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7680cc070 con 0x7fe7680cb900 2024-04-12T19:50:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.528+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76815e710 con 0x7fe7680cb900 2024-04-12T19:50:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.529+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe76001a070 con 0x7fe7680cb900 2024-04-12T19:50:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.530+0000 7fe75f400640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 0x7fe74406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.530+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fe76008aba0 con 0x7fe7680cb900 2024-04-12T19:50:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.530+0000 7fe766a00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 0x7fe74406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.531+0000 7fe766a00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 0x7fe74406ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe758000f60 tx=0x7fe758009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.534+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe76815e710 con 0x7fe7680cb900 2024-04-12T19:50:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.785+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7680b4b90 con 0x7fe7680cb900 2024-04-12T19:50:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.787+0000 7fe75f400640 1 -- 172.21.15.32:0/1688911045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7fe760058cc0 con 0x7fe7680cb900 2024-04-12T19:50:19.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:19.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-12T19:50:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-04-12T19:50:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.789+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 msgr2=0x7fe74406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.789+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 0x7fe74406ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe758000f60 tx=0x7fe758009040 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.789+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 msgr2=0x7fe768158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.789+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c430 tx=0x7fe76000c8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 shutdown_connections 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe74406c920 0x7fe74406ede0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7680cb900 0x7fe768158500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76815ebb0 0x7fe76815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 --2- 172.21.15.32:0/1688911045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680d1af0 0x7fe7680b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 >> 172.21.15.32:0/1688911045 conn(0x7fe76806d040 msgr2=0x7fe76806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 shutdown_connections 2024-04-12T19:50:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:19.790+0000 7fe767e00640 1 -- 172.21.15.32:0/1688911045 wait complete. 2024-04-12T19:50:19.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.029+0000 7f8423200640 1 Processor -- start 2024-04-12T19:50:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.029+0000 7f8423200640 1 -- start start 2024-04-12T19:50:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.030+0000 7f8423200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8424162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.030+0000 7f8423200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84240d0090 con 0x7f8424162020 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.030+0000 7f8423200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84240cfad0 con 0x7f84240cf110 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.030+0000 7f8423200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8424163440 con 0x7f84240d1c90 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.031+0000 7f8421e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8424162020 0x7f84240d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51154/0 (socket says 172.21.15.32:51154) 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.031+0000 7f8421e00640 1 -- 172.21.15.32:0/3893254333 learned_addr learned my addr 172.21.15.32:0/3893254333 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.031+0000 7f8421400640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8424162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.031+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183909918 0 0) 0x7f8414001490 con 0x7f8424162020 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.031+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8404003520 con 0x7f8424162020 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3678540814 0 0) 0x7f8404003520 con 0x7f8424162020 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 msgr2=0x7f8424162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8424162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 >> v1:172.21.15.169:6789/0 conn(0x7f84240cf110 legacy=0x7f84240d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84240cfdb0 con 0x7f8424162020 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f8414001bc0 con 0x7f8424162020 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 >> v1:172.21.15.32:6789/0 conn(0x7f8424162020 legacy=0x7f84240d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040039e0 0x7f8404003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8404004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8404004d40 0x7f84040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8424163440 con 0x7f84040039e0 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f840400baa0 con 0x7f8404004d40 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f840400bca0 con 0x7f84240d1c90 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040039e0 0x7f8404003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421400640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8404004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8422800640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8404004d40 0x7f84040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 -- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 msgr2=0x7f8404004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84240d1c90 0x7f8404004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 -- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8404004d40 msgr2=0x7f84040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8404004d40 0x7f84040051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.032+0000 7f8421e00640 1 -- 172.21.15.32:0/3893254333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8404003520 con 0x7f84040039e0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.033+0000 7f8421e00640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040039e0 0x7f8404003e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8414002230 tx=0x7f8404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0bae27a18042f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400a5a0 con 0x7f84040039e0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8414034650 con 0x7f84040039e0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8420a00640 1 -- 172.21.15.32:0/3893254333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414034a40 con 0x7f84040039e0 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8423200640 1 -- 172.21.15.32:0/3893254333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040039e0 msgr2=0x7f8404003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8423200640 1 --2- 172.21.15.32:0/3893254333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84040039e0 0x7f8404003e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8414002230 tx=0x7f8404005830 comp rx=0 tx=0).stop 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8422800640 1 -- 172.21.15.32:0/3893254333 reap_dead start 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8423200640 1 -- 172.21.15.32:0/3893254333 shutdown_connections 2024-04-12T19:50:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8423200640 1 -- 172.21.15.32:0/3893254333 >> 172.21.15.32:0/3893254333 conn(0x7f842406cf00 msgr2=0x7f842406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.034+0000 7f8423200640 1 -- 172.21.15.32:0/3893254333 shutdown_connections 2024-04-12T19:50:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.035+0000 7f8423200640 1 -- 172.21.15.32:0/3893254333 wait complete. 2024-04-12T19:50:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.036+0000 7f8423200640 1 Processor -- start 2024-04-12T19:50:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.036+0000 7f8423200640 1 -- start start 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842415cdf0 0x7f84240c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842415a5b0 0x7f842415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84240cf880 con 0x7f842415d1d0 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84240b7b30 con 0x7f842415a5b0 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8423200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84240d2e70 con 0x7f842415cdf0 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8421400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8421400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55202/0 (socket says 172.21.15.32:55202) 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.037+0000 7f8421400640 1 -- 172.21.15.32:0/3021205016 learned_addr learned my addr 172.21.15.32:0/3021205016 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 -- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842415cdf0 msgr2=0x7f84240c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842415cdf0 0x7f84240c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 -- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842415a5b0 msgr2=0x7f842415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842415a5b0 0x7f842415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 -- 172.21.15.32:0/3021205016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842415b2c0 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8421400640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f840c011590 tx=0x7f840c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c004220 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c0043c0 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c015870 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.038+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842415b5a0 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.039+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8424144d10 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.040+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84240755c0 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.040+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f840c015a10 con 0x7f842415d1d0 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.041+0000 7f841a000640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 0x7f840006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.041+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f840c08bd80 con 0x7f842415d1d0 2024-04-12T19:50:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.042+0000 7f8421e00640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 0x7f840006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.042+0000 7f8421e00640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 0x7f840006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8414034000 tx=0x7f8414009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.046+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f840c019030 con 0x7f842415d1d0 2024-04-12T19:50:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.246+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84240b7650 con 0x7f842415d1d0 2024-04-12T19:50:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.248+0000 7f841a000640 1 -- 172.21.15.32:0/3021205016 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12872 (secure 0 0 0) 0x7f840c05a4f0 con 0x7f842415d1d0 2024-04-12T19:50:20.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:20.249 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:14.442467+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 msgr2=0x7f840006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 0x7f840006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8414034000 tx=0x7f8414009030 comp rx=0 tx=0).stop 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 msgr2=0x7f84240c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f840c011590 tx=0x7f840c011a50 comp rx=0 tx=0).stop 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 shutdown_connections 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f840006c8f0 0x7f840006edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842415a5b0 0x7f842415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f842415d1d0 0x7f84240c7340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 --2- 172.21.15.32:0/3021205016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842415cdf0 0x7f84240c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 >> 172.21.15.32:0/3021205016 conn(0x7f842406cf00 msgr2=0x7f84240c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:20.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 shutdown_connections 2024-04-12T19:50:20.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.251+0000 7f8423200640 1 -- 172.21.15.32:0/3021205016 wait complete. 2024-04-12T19:50:20.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:20.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.497+0000 7efdf3200640 1 Processor -- start 2024-04-12T19:50:20.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.497+0000 7efdf3200640 1 -- start start 2024-04-12T19:50:20.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf4161f80 0x7efdf40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdf40cfd20 0x7efdf40b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdf40d26b0 0x7efdf40d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efdf40d2bb0 con 0x7efdf40d26b0 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efdf40d2e90 con 0x7efdf40cfd20 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdf40d3170 con 0x7efdf4161f80 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdf40d26b0 0x7efdf40d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdf40d26b0 0x7efdf40d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55214/0 (socket says 172.21.15.32:55214) 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf2800640 1 -- 172.21.15.32:0/2964129311 learned_addr learned my addr 172.21.15.32:0/2964129311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf1400640 1 --2- 172.21.15.32:0/2964129311 >> v2:172.21.15.169:3300/0 conn(0x7efdf40cfd20 0x7efdf40b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.499+0000 7efdf1e00640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf4161f80 0x7efdf40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.500+0000 7efdf2800640 1 -- 172.21.15.32:0/2964129311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf4161f80 msgr2=0x7efdf40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.500+0000 7efdf2800640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf4161f80 0x7efdf40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.500+0000 7efdf2800640 1 -- 172.21.15.32:0/2964129311 >> v2:172.21.15.169:3300/0 conn(0x7efdf40cfd20 msgr2=0x7efdf40b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.500+0000 7efdf2800640 1 --2- 172.21.15.32:0/2964129311 >> v2:172.21.15.169:3300/0 conn(0x7efdf40cfd20 0x7efdf40b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.500+0000 7efdf2800640 1 -- 172.21.15.32:0/2964129311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf40d3450 con 0x7efdf40d26b0 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.501+0000 7efdf2800640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf40d26b0 0x7efdf40d1580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdf40d5840 tx=0x7efdec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cd5f801b3f85c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.501+0000 7efdf0a00640 1 -- 172.21.15.32:0/2964129311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec02ca70 con 0x7efdf40d26b0 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.501+0000 7efdf0a00640 1 -- 172.21.15.32:0/2964129311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec02cc10 con 0x7efdf40d26b0 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.501+0000 7efdf0a00640 1 -- 172.21.15.32:0/2964129311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0343f0 con 0x7efdf40d26b0 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 -- 172.21.15.32:0/2964129311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf40d26b0 msgr2=0x7efdf40d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf40d26b0 0x7efdf40d1580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efdf40d5840 tx=0x7efdec02bed0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 -- 172.21.15.32:0/2964129311 shutdown_connections 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf40d26b0 0x7efdf40d1580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 --2- 172.21.15.32:0/2964129311 >> v2:172.21.15.169:3300/0 conn(0x7efdf40cfd20 0x7efdf40b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 --2- 172.21.15.32:0/2964129311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf4161f80 0x7efdf40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 -- 172.21.15.32:0/2964129311 >> 172.21.15.32:0/2964129311 conn(0x7efdf406ce40 msgr2=0x7efdf406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:20.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 -- 172.21.15.32:0/2964129311 shutdown_connections 2024-04-12T19:50:20.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.502+0000 7efdf3200640 1 -- 172.21.15.32:0/2964129311 wait complete. 2024-04-12T19:50:20.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.504+0000 7efdf3200640 1 Processor -- start 2024-04-12T19:50:20.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.504+0000 7efdf3200640 1 -- start start 2024-04-12T19:50:20.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf40cfd20 0x7efdf4160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 0x7efdf4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efdf40b8000 con 0x7efdf4161f80 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdf40b7e80 con 0x7efdf40d26b0 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdf40b8180 con 0x7efdf40cfd20 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55222/0 (socket says 172.21.15.32:55222) 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 -- 172.21.15.32:0/3863496781 learned_addr learned my addr 172.21.15.32:0/3863496781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf1400640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 0x7efdf4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf1e00640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf40cfd20 0x7efdf4160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 -- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf40cfd20 msgr2=0x7efdf4160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf40cfd20 0x7efdf4160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 -- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 msgr2=0x7efdf4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 0x7efdf4158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf2800640 1 -- 172.21.15.32:0/3863496781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf41f8400 con 0x7efdf4161f80 2024-04-12T19:50:20.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.505+0000 7efdf1400640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 0x7efdf4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:20.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdf2800640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efdec009f40 tx=0x7efdec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec043020 con 0x7efdf4161f80 2024-04-12T19:50:20.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec03e450 con 0x7efdf4161f80 2024-04-12T19:50:20.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf41f86e0 con 0x7efdf4161f80 2024-04-12T19:50:20.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec009510 con 0x7efdf4161f80 2024-04-12T19:50:20.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.506+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41f8ca0 con 0x7efdf4161f80 2024-04-12T19:50:20.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.507+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf40755c0 con 0x7efdf4161f80 2024-04-12T19:50:20.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.508+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdec046020 con 0x7efdf4161f80 2024-04-12T19:50:20.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.509+0000 7efdea000640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 0x7efdd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.509+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7efdec0adfe0 con 0x7efdf4161f80 2024-04-12T19:50:20.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.510+0000 7efdf1e00640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 0x7efdd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.510+0000 7efdf1e00640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 0x7efdd406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.514+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdec04d030 con 0x7efdf4161f80 2024-04-12T19:50:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.722+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efdf4068ef0 con 0x7efdf4161f80 2024-04-12T19:50:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.723+0000 7efdea000640 1 -- 172.21.15.32:0/3863496781 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7efdec07c7b0 con 0x7efdf4161f80 2024-04-12T19:50:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 msgr2=0x7efdd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 0x7efdd406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4008040 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 msgr2=0x7efdf4152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efdec009f40 tx=0x7efdec004a40 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 shutdown_connections 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdd406c8f0 0x7efdd406edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4161f80 0x7efdf4152a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdf40d26b0 0x7efdf4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 --2- 172.21.15.32:0/3863496781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf40cfd20 0x7efdf4160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 >> 172.21.15.32:0/3863496781 conn(0x7efdf406ce40 msgr2=0x7efdf4156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 shutdown_connections 2024-04-12T19:50:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.726+0000 7efdf3200640 1 -- 172.21.15.32:0/3863496781 wait complete. 2024-04-12T19:50:20.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-12T19:50:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.965+0000 7fc10f200640 1 Processor -- start 2024-04-12T19:50:20.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.966+0000 7fc10f200640 1 -- start start 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc11015ebb0 0x7fc110163060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10f200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1101635a0 con 0x7fc1100d1cc0 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11015ef90 con 0x7fc1100b3c70 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc1100cf090 con 0x7fc11015ebb0 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1100d1cc0 0x7fc1100d2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51160/0 (socket says 172.21.15.32:51160) 2024-04-12T19:50:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10e800640 1 -- 172.21.15.32:0/1947962924 learned_addr learned my addr 172.21.15.32:0/1947962924 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.967+0000 7fc10de00640 1 --2- 172.21.15.32:0/1947962924 >> v2:172.21.15.169:3300/0 conn(0x7fc11015ebb0 0x7fc110163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724724267 0 0) 0x7fc1101635a0 con 0x7fc1100d1cc0 2024-04-12T19:50:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f4003520 con 0x7fc1100d1cc0 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3115429311 0 0) 0x7fc0f4003520 con 0x7fc1100d1cc0 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 >> v2:172.21.15.169:3300/0 conn(0x7fc11015ebb0 msgr2=0x7fc110163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 --2- 172.21.15.32:0/1947962924 >> v2:172.21.15.169:3300/0 conn(0x7fc11015ebb0 0x7fc110163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 >> v1:172.21.15.169:6789/0 conn(0x7fc1100b3c70 legacy=0x7fc1100d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100cf370 con 0x7fc1100d1cc0 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fc1080019f0 con 0x7fc1100d1cc0 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 >> v1:172.21.15.32:6789/0 conn(0x7fc1100d1cc0 legacy=0x7fc1100d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f4003a00 0x7fc0f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4004800 0x7fc0f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 0x7fc0f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100cf090 con 0x7fc0f40051c0 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f400c0c0 con 0x7fc0f4004800 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.968+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f400c2c0 con 0x7fc0f4003a00 2024-04-12T19:50:20.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 0x7fc0f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10d400640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4004800 0x7fc0f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10de00640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f4003a00 0x7fc0f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 -- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f4003a00 msgr2=0x7fc0f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f4003a00 0x7fc0f4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 -- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4004800 msgr2=0x7fc0f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0f4004800 0x7fc0f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.969+0000 7fc10e800640 1 -- 172.21.15.32:0/1947962924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f4003520 con 0x7fc0f40051c0 2024-04-12T19:50:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.970+0000 7fc10e800640 1 -- 172.21.15.32:0/1947962924 reap_dead start 2024-04-12T19:50:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.970+0000 7fc10e800640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 0x7fc0f400b9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc10800a790 tx=0x7fc0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806cc782c98bc27f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800a3b0 con 0x7fc0f40051c0 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc108034670 con 0x7fc0f40051c0 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10ca00640 1 -- 172.21.15.32:0/1947962924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108034a60 con 0x7fc0f40051c0 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 -- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 msgr2=0x7fc0f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 0x7fc0f400b9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc10800a790 tx=0x7fc0f4005830 comp rx=0 tx=0).stop 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 -- 172.21.15.32:0/1947962924 shutdown_connections 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 --2- 172.21.15.32:0/1947962924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0f40051c0 0x7fc0f400b9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 -- 172.21.15.32:0/1947962924 >> 172.21.15.32:0/1947962924 conn(0x7fc11006d040 msgr2=0x7fc11006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 -- 172.21.15.32:0/1947962924 shutdown_connections 2024-04-12T19:50:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.971+0000 7fc10f200640 1 -- 172.21.15.32:0/1947962924 wait complete. 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.973+0000 7fc10f200640 1 Processor -- start 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.973+0000 7fc10f200640 1 -- start start 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 0x7fc110151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 0x7fc1100b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100cfc10 con 0x7fc110154060 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100cf930 con 0x7fc110152780 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100cfef0 con 0x7fc0f40051c0 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55250/0 (socket says 172.21.15.32:55250) 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.974+0000 7fc10d400640 1 -- 172.21.15.32:0/648085615 learned_addr learned my addr 172.21.15.32:0/648085615 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10de00640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 0x7fc110151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10e800640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 0x7fc1100b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 -- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 msgr2=0x7fc110151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 0x7fc110151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 -- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 msgr2=0x7fc1100b9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 0x7fc1100b9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 -- 172.21.15.32:0/648085615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100b65a0 con 0x7fc110154060 2024-04-12T19:50:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10e800640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 0x7fc1100b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10de00640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 0x7fc110151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.975+0000 7fc10d400640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00e440 tx=0x7fc0fc00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.976+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0052e0 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.976+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc018070 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.976+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc003810 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.976+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1100b6880 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.976+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1100b6e40 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.977+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1100755c0 con 0x7fc110154060 2024-04-12T19:50:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.978+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0fc0039b0 con 0x7fc110154060 2024-04-12T19:50:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.979+0000 7fc106000640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.979+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc0fc08b420 con 0x7fc110154060 2024-04-12T19:50:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.979+0000 7fc10de00640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.980+0000 7fc10de00640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1000099e0 tx=0x7fc100008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:20.983+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0fc059ba0 con 0x7fc110154060 2024-04-12T19:50:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.191+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc1100d2540 con 0x7fc110154060 2024-04-12T19:50:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.192+0000 7fc106000640 1 -- 172.21.15.32:0/648085615 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fc0fc0594a0 con 0x7fc110154060 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:50:21.193 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 msgr2=0x7fc0f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1000099e0 tx=0x7fc100008040 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 msgr2=0x7fc1100b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00e440 tx=0x7fc0fc00e900 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 shutdown_connections 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc110152780 0x7fc1100b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110154060 0x7fc1100b8ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 --2- 172.21.15.32:0/648085615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0f40051c0 0x7fc110151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.195+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 >> 172.21.15.32:0/648085615 conn(0x7fc11006d040 msgr2=0x7fc1100b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.196+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 shutdown_connections 2024-04-12T19:50:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.196+0000 7fc10f200640 1 -- 172.21.15.32:0/648085615 wait complete. 2024-04-12T19:50:21.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-12T19:50:21.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-12T19:50:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.443+0000 7f3131a00640 1 Processor -- start 2024-04-12T19:50:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.443+0000 7f3131a00640 1 -- start start 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f3131a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f312c0b3760 0x7f312c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f3131a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 0x7f312c072e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f3131a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c0b5480 con 0x7f312c0afd10 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f3131a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f312c06e160 con 0x7f312c0b3760 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f3131a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f312c0b3cb0 con 0x7f312c0729e0 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f312c0b3760 0x7f312c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f312c0b3760 0x7f312c15bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55256/0 (socket says 172.21.15.32:55256) 2024-04-12T19:50:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 -- 172.21.15.32:0/1316471141 learned_addr learned my addr 172.21.15.32:0/1316471141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312b400640 1 --2- 172.21.15.32:0/1316471141 >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 0x7f312c072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 -- 172.21.15.32:0/1316471141 >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 msgr2=0x7f312c072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 --2- 172.21.15.32:0/1316471141 >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 0x7f312c072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 -- 172.21.15.32:0/1316471141 >> v1:172.21.15.169:6789/0 conn(0x7f312c0afd10 legacy=0x7f312c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312a000640 1 -- 172.21.15.32:0/1316471141 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c0b3f90 con 0x7f312c0b3760 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.445+0000 7f312b400640 1 --2- 172.21.15.32:0/1316471141 >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 0x7f312c072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.447+0000 7f312a000640 1 --2- 172.21.15.32:0/1316471141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0b3760 0x7f312c15bf50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f312c069d10 tx=0x7f311802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c12e9a90b1e4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.447+0000 7f3129600640 1 -- 172.21.15.32:0/1316471141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118035020 con 0x7f312c0b3760 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.447+0000 7f3129600640 1 -- 172.21.15.32:0/1316471141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3118031870 con 0x7f312c0b3760 2024-04-12T19:50:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.447+0000 7f3129600640 1 -- 172.21.15.32:0/1316471141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118031c40 con 0x7f312c0b3760 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 -- 172.21.15.32:0/1316471141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0b3760 msgr2=0x7f312c15bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 --2- 172.21.15.32:0/1316471141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0b3760 0x7f312c15bf50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f312c069d10 tx=0x7f311802b910 comp rx=0 tx=0).stop 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 -- 172.21.15.32:0/1316471141 shutdown_connections 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 --2- 172.21.15.32:0/1316471141 >> v2:172.21.15.169:3300/0 conn(0x7f312c0729e0 0x7f312c072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 --2- 172.21.15.32:0/1316471141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0b3760 0x7f312c15bf50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 -- 172.21.15.32:0/1316471141 >> 172.21.15.32:0/1316471141 conn(0x7f312c0b2460 msgr2=0x7f312c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:21.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 -- 172.21.15.32:0/1316471141 shutdown_connections 2024-04-12T19:50:21.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.448+0000 7f3131a00640 1 -- 172.21.15.32:0/1316471141 wait complete. 2024-04-12T19:50:21.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.450+0000 7f3131a00640 1 Processor -- start 2024-04-12T19:50:21.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.450+0000 7f3131a00640 1 -- start start 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 0x7f312c1f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 0x7f312c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0b00f0 con 0x7f312c0afd10 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0b4550 con 0x7f312c0b3760 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f3131a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f312c072150 con 0x7f312c0729e0 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55258/0 (socket says 172.21.15.32:55258) 2024-04-12T19:50:21.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 -- 172.21.15.32:0/310434599 learned_addr learned my addr 172.21.15.32:0/310434599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312aa00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 0x7f312c1f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312b400640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 0x7f312c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 -- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 msgr2=0x7f312c1f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:21.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 0x7f312c1f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 -- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 msgr2=0x7f312c147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 0x7f312c147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.451+0000 7f312a000640 1 -- 172.21.15.32:0/310434599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1f87b0 con 0x7f312c0afd10 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f312aa00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 0x7f312c1f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f312b400640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 0x7f312c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f312a000640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f311802b950 tx=0x7f31180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118041070 con 0x7f312c0afd10 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311803b430 con 0x7f312c0afd10 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118035040 con 0x7f312c0afd10 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1f8a90 con 0x7f312c0afd10 2024-04-12T19:50:21.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.452+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c0adb00 con 0x7f312c0afd10 2024-04-12T19:50:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.454+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c1f8fb0 con 0x7f312c0afd10 2024-04-12T19:50:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.454+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f311804c060 con 0x7f312c0afd10 2024-04-12T19:50:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.455+0000 7f3122a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 0x7f310c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:21.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.455+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f31180ad910 con 0x7f312c0afd10 2024-04-12T19:50:21.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.456+0000 7f312aa00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 0x7f310c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:21.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.456+0000 7f312aa00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 0x7f310c06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f311c007fd0 tx=0x7f311c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.460+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f312c1f8fb0 con 0x7f312c0afd10 2024-04-12T19:50:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:21.663+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f312c068db0 con 0x7f312c0afd10 2024-04-12T19:50:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.631+0000 7f3122a00640 1 -- 172.21.15.32:0/310434599 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f311807bff0 con 0x7f312c0afd10 2024-04-12T19:50:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-12T19:50:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.634+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 msgr2=0x7f310c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 0x7f310c06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f311c007fd0 tx=0x7f311c009040 comp rx=0 tx=0).stop 2024-04-12T19:50:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 msgr2=0x7f312c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f311802b950 tx=0x7f31180046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 shutdown_connections 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f310c06c8f0 0x7f310c06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0b3760 0x7f312c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f312c0afd10 0x7f312c159140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 --2- 172.21.15.32:0/310434599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0729e0 0x7f312c1f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 >> 172.21.15.32:0/310434599 conn(0x7f312c0b2460 msgr2=0x7f312c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 shutdown_connections 2024-04-12T19:50:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.635+0000 7f3131a00640 1 -- 172.21.15.32:0/310434599 wait complete. 2024-04-12T19:50:22.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-12T19:50:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.877+0000 7f7806c00640 1 Processor -- start 2024-04-12T19:50:22.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.878+0000 7f7806c00640 1 -- start start 2024-04-12T19:50:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7806c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7800162030 0x7f78000b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7806c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f78000d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7806c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78000cffe0 con 0x7f78000cf800 2024-04-12T19:50:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7806c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f78000cfd00 con 0x7f7800162030 2024-04-12T19:50:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7806c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78000d3040 con 0x7f780006d420 2024-04-12T19:50:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.879+0000 7f7804e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78000cf800 0x7f780006cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51176/0 (socket says 172.21.15.32:51176) 2024-04-12T19:50:22.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f7804e00640 1 -- 172.21.15.32:0/3671527645 learned_addr learned my addr 172.21.15.32:0/3671527645 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531554192 0 0) 0x7f78000cffe0 con 0x7f78000cf800 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77e8003520 con 0x7f78000cf800 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> v2:172.21.15.169:3300/0 conn(0x7f7800162030 0x7f78000b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f7806200640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f78000d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2806652304 0 0) 0x7f77e8003520 con 0x7f78000cf800 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 msgr2=0x7f78000d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f78000d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 >> v2:172.21.15.169:3300/0 conn(0x7f7800162030 msgr2=0x7f78000b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.880+0000 7f77ffe00640 1 --2- 172.21.15.32:0/3671527645 >> v2:172.21.15.169:3300/0 conn(0x7f7800162030 0x7f78000b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78000d3320 con 0x7f78000cf800 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f77ec001670 con 0x7f78000cf800 2024-04-12T19:50:22.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> v2:172.21.15.169:3300/0 conn(0x7f7800162030 0x7f78000b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7806200640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f78000d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 >> v1:172.21.15.32:6789/0 conn(0x7f78000cf800 legacy=0x7f780006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77e8003a20 0x7f77e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e8004800 0x7f77e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f77e8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78000cfd00 con 0x7f77e8003a20 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f78000d3040 con 0x7f77e8004800 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77e800bbd0 con 0x7f780006d420 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77e8003a20 0x7f77e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7804e00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e8004800 0x7f77e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 -- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 msgr2=0x7f77e8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f780006d420 0x7f77e8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 -- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e8004800 msgr2=0x7f77e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:22.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e8004800 0x7f77e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.881+0000 7f7805800640 1 -- 172.21.15.32:0/3671527645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e8003520 con 0x7f77e8003a20 2024-04-12T19:50:22.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.882+0000 7f7804e00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77e8004800 0x7f77e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:22.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.883+0000 7f7805800640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77e8003a20 0x7f77e8003e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77f40098a0 tx=0x7f77e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd899c8ad8f4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.883+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f400f040 con 0x7f77e8003a20 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.883+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f4004230 con 0x7f77e8003a20 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.883+0000 7f77ffe00640 1 -- 172.21.15.32:0/3671527645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4003610 con 0x7f77e8003a20 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 -- 172.21.15.32:0/3671527645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77e8003a20 msgr2=0x7f77e8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 --2- 172.21.15.32:0/3671527645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77e8003a20 0x7f77e8003e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77f40098a0 tx=0x7f77e8005830 comp rx=0 tx=0).stop 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806200640 1 -- 172.21.15.32:0/3671527645 reap_dead start 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 -- 172.21.15.32:0/3671527645 shutdown_connections 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 -- 172.21.15.32:0/3671527645 >> 172.21.15.32:0/3671527645 conn(0x7f78000d1970 msgr2=0x7f78000d48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 -- 172.21.15.32:0/3671527645 shutdown_connections 2024-04-12T19:50:22.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.884+0000 7f7806c00640 1 -- 172.21.15.32:0/3671527645 wait complete. 2024-04-12T19:50:22.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.886+0000 7f7806c00640 1 Processor -- start 2024-04-12T19:50:22.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.886+0000 7f7806c00640 1 -- start start 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 0x7f78000abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 0x7f78000ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78000b7ec0 con 0x7f78000ae4f0 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7800162410 con 0x7f78000adb30 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7806c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78000b8040 con 0x7f78000ac0b0 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.887+0000 7f7804e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55272/0 (socket says 172.21.15.32:55272) 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 -- 172.21.15.32:0/2369434670 learned_addr learned my addr 172.21.15.32:0/2369434670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7805800640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 0x7f78000abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7806200640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 0x7f78000ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 -- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 msgr2=0x7f78000ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 0x7f78000ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 -- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 msgr2=0x7f78000abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 0x7f78000abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 -- 172.21.15.32:0/2369434670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001f5240 con 0x7f78000ae4f0 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7805800640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 0x7f78000abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7806200640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 0x7f78000ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.888+0000 7f7804e00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77ec000ff0 tx=0x7f77ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.889+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec019070 con 0x7f78000ae4f0 2024-04-12T19:50:22.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.889+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ec004cb0 con 0x7f78000ae4f0 2024-04-12T19:50:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.889+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec004e60 con 0x7f78000ae4f0 2024-04-12T19:50:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.889+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78001f5520 con 0x7f78000ae4f0 2024-04-12T19:50:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.889+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78000adf10 con 0x7f78000ae4f0 2024-04-12T19:50:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.890+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78000755c0 con 0x7f78000ae4f0 2024-04-12T19:50:22.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.891+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77ec0072f0 con 0x7f78000ae4f0 2024-04-12T19:50:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.892+0000 7f77fd600640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 0x7f77e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.892+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f77ec08b490 con 0x7f78000ae4f0 2024-04-12T19:50:22.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.892+0000 7f7805800640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 0x7f77e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:22.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.893+0000 7f7805800640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 0x7f77e006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77f4035000 tx=0x7f77f4039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:22.897+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77ec059a90 con 0x7f78000ae4f0 2024-04-12T19:50:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.101+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f780006dac0 con 0x7f78000ae4f0 2024-04-12T19:50:23.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.646+0000 7f77fd600640 1 -- 172.21.15.32:0/2369434670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f77ec059390 con 0x7f78000ae4f0 2024-04-12T19:50:23.647 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-12T19:50:23.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 msgr2=0x7f77e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 0x7f77e006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77f4035000 tx=0x7f77f4039040 comp rx=0 tx=0).stop 2024-04-12T19:50:23.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 msgr2=0x7f78001f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77ec000ff0 tx=0x7f77ec00dd40 comp rx=0 tx=0).stop 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 shutdown_connections 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.651+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f77e006c8f0 0x7f77e006edb0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77f4035000 tx=0x7f77f4039040 comp rx=0 tx=0).stop 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.652+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000ac0b0 0x7f78000ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.652+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78000ae4f0 0x7f78001f4850 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.652+0000 7f7806c00640 1 --2- 172.21.15.32:0/2369434670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78000adb30 0x7f78000abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.652+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 >> 172.21.15.32:0/2369434670 conn(0x7f78000d1970 msgr2=0x7f7800160800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.653+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 shutdown_connections 2024-04-12T19:50:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.656+0000 7f7806c00640 1 -- 172.21.15.32:0/2369434670 wait complete. 2024-04-12T19:50:23.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-12T19:50:23.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.887+0000 7f433e000640 1 Processor -- start 2024-04-12T19:50:23.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.887+0000 7f433e000640 1 -- start start 2024-04-12T19:50:23.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f433e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f433815a440 0x7f433815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f433e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f433806fc80 0x7f43380cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f433e000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433815d520 con 0x7f43380cd620 2024-04-12T19:50:23.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f433e000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f433806e280 con 0x7f433815a440 2024-04-12T19:50:23.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f433e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f433815d800 con 0x7f433806fc80 2024-04-12T19:50:23.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f4337e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f43380cd620 0x7f4338072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43184/0 (socket says 172.21.15.32:43184) 2024-04-12T19:50:23.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.889+0000 7f4337e00640 1 -- 172.21.15.32:0/490986984 learned_addr learned my addr 172.21.15.32:0/490986984 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:23.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4337400640 1 --2- 172.21.15.32:0/490986984 >> v2:172.21.15.32:3300/0 conn(0x7f433815a440 0x7f433815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f432ea00640 1 --2- 172.21.15.32:0/490986984 >> v2:172.21.15.169:3300/0 conn(0x7f433806fc80 0x7f43380cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4336a00640 1 -- 172.21.15.32:0/490986984 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401743746 0 0) 0x7f433815d520 con 0x7f43380cd620 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4336a00640 1 -- 172.21.15.32:0/490986984 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f431c003520 con 0x7f43380cd620 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4337400640 1 -- 172.21.15.32:0/490986984 >> v2:172.21.15.169:3300/0 conn(0x7f433806fc80 msgr2=0x7f43380cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4337400640 1 --2- 172.21.15.32:0/490986984 >> v2:172.21.15.169:3300/0 conn(0x7f433806fc80 0x7f43380cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4337400640 1 -- 172.21.15.32:0/490986984 >> v1:172.21.15.32:6789/0 conn(0x7f43380cd620 legacy=0x7f4338072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.890+0000 7f4337400640 1 -- 172.21.15.32:0/490986984 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433815dae0 con 0x7f433815a440 2024-04-12T19:50:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.891+0000 7f4337400640 1 --2- 172.21.15.32:0/490986984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f433815cef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43380ccb70 tx=0x7f432802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5544aa5ecbd4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f4336a00640 1 -- 172.21.15.32:0/490986984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280315a0 con 0x7f433815a440 2024-04-12T19:50:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f4336a00640 1 -- 172.21.15.32:0/490986984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4328037070 con 0x7f433815a440 2024-04-12T19:50:23.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f4336a00640 1 -- 172.21.15.32:0/490986984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328031940 con 0x7f433815a440 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 -- 172.21.15.32:0/490986984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 msgr2=0x7f433815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 --2- 172.21.15.32:0/490986984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f433815cef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43380ccb70 tx=0x7f432802b840 comp rx=0 tx=0).stop 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 -- 172.21.15.32:0/490986984 shutdown_connections 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 --2- 172.21.15.32:0/490986984 >> v2:172.21.15.169:3300/0 conn(0x7f433806fc80 0x7f43380cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 --2- 172.21.15.32:0/490986984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f433815cef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.892+0000 7f433e000640 1 -- 172.21.15.32:0/490986984 >> 172.21.15.32:0/490986984 conn(0x7f43380b2460 msgr2=0x7f43380b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.893+0000 7f433e000640 1 -- 172.21.15.32:0/490986984 shutdown_connections 2024-04-12T19:50:23.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.893+0000 7f433e000640 1 -- 172.21.15.32:0/490986984 wait complete. 2024-04-12T19:50:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.894+0000 7f433e000640 1 Processor -- start 2024-04-12T19:50:23.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.894+0000 7f433e000640 1 -- start start 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f433806fc80 0x7f43380cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380cd620 0x7f4338135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f433806f510 con 0x7f433815a440 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f433815ddc0 con 0x7f433806fc80 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.895+0000 7f433e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4338072860 con 0x7f43380cd620 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60816/0 (socket says 172.21.15.32:60816) 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 -- 172.21.15.32:0/3710158067 learned_addr learned my addr 172.21.15.32:0/3710158067 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337400640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f433806fc80 0x7f43380cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f432ea00640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380cd620 0x7f4338135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 -- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380cd620 msgr2=0x7f4338135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380cd620 0x7f4338135f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 -- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f433806fc80 msgr2=0x7f43380cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f433806fc80 0x7f43380cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 -- 172.21.15.32:0/3710158067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380ab250 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.896+0000 7f4337e00640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f433000cbe0 tx=0x7f433000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.897+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017070 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.897+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433000ad40 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.897+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330012600 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.897+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380cc6c0 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.897+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4338134300 con 0x7f433815a440 2024-04-12T19:50:23.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.898+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4338068c80 con 0x7f433815a440 2024-04-12T19:50:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.899+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f433001a070 con 0x7f433815a440 2024-04-12T19:50:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.900+0000 7f432fe00640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 0x7f431406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.900+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f433008b770 con 0x7f433815a440 2024-04-12T19:50:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.900+0000 7f4337400640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 0x7f431406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.901+0000 7f4337400640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 0x7f431406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4328036000 tx=0x7f432803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:23.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:23.906+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43300595a0 con 0x7f433815a440 2024-04-12T19:50:24.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.147+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f43380cfd80 con 0x7f433815a440 2024-04-12T19:50:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.658+0000 7f432fe00640 1 -- 172.21.15.32:0/3710158067 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f433005d8a0 con 0x7f433815a440 2024-04-12T19:50:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-04-12T19:50:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.664+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 msgr2=0x7f431406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.664+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 0x7f431406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4328036000 tx=0x7f432803d040 comp rx=0 tx=0).stop 2024-04-12T19:50:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.664+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 msgr2=0x7f43381364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.664+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f433000cbe0 tx=0x7f433000a430 comp rx=0 tx=0).stop 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 shutdown_connections 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f431406c920 0x7f431406ede0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4328036000 tx=0x7f432803d040 comp rx=0 tx=0).stop 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433815a440 0x7f43381364c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43380cd620 0x7f4338135f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 --2- 172.21.15.32:0/3710158067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f433806fc80 0x7f43380cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.665+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 >> 172.21.15.32:0/3710158067 conn(0x7f43380b2460 msgr2=0x7f43380ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.667+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 shutdown_connections 2024-04-12T19:50:24.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.667+0000 7f433e000640 1 -- 172.21.15.32:0/3710158067 wait complete. 2024-04-12T19:50:24.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.892+0000 7fd38f200640 1 Processor -- start 2024-04-12T19:50:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.892+0000 7fd38f200640 1 -- start start 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3900cee00 0x7fd39006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd390071f50 0x7fd3900723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd39015cbe0 0x7fd39015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd39006f4f0 con 0x7fd3900cee00 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd3900728f0 con 0x7fd39015cbe0 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.893+0000 7fd38f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd39015dbb0 con 0x7fd390071f50 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3900cee00 0x7fd39006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3900cee00 0x7fd39006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60824/0 (socket says 172.21.15.32:60824) 2024-04-12T19:50:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 -- 172.21.15.32:0/4047992824 learned_addr learned my addr 172.21.15.32:0/4047992824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd387e00640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd390071f50 0x7fd3900723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 -- 172.21.15.32:0/4047992824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd390071f50 msgr2=0x7fd3900723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd390071f50 0x7fd3900723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 -- 172.21.15.32:0/4047992824 >> v2:172.21.15.169:3300/0 conn(0x7fd39015cbe0 msgr2=0x7fd39015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 --2- 172.21.15.32:0/4047992824 >> v2:172.21.15.169:3300/0 conn(0x7fd39015cbe0 0x7fd39015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.894+0000 7fd38de00640 1 -- 172.21.15.32:0/4047992824 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39015d8d0 con 0x7fd3900cee00 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.895+0000 7fd38de00640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3900cee00 0x7fd39006e8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd380009870 tx=0x7fd38002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608cdd277de4cf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38d400640 1 -- 172.21.15.32:0/4047992824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380035020 con 0x7fd3900cee00 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38d400640 1 -- 172.21.15.32:0/4047992824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd380031640 con 0x7fd3900cee00 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38d400640 1 -- 172.21.15.32:0/4047992824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38002d040 con 0x7fd3900cee00 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 -- 172.21.15.32:0/4047992824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3900cee00 msgr2=0x7fd39006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3900cee00 0x7fd39006e8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd380009870 tx=0x7fd38002b8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 -- 172.21.15.32:0/4047992824 shutdown_connections 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 --2- 172.21.15.32:0/4047992824 >> v2:172.21.15.169:3300/0 conn(0x7fd39015cbe0 0x7fd39015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd390071f50 0x7fd3900723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 --2- 172.21.15.32:0/4047992824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3900cee00 0x7fd39006e8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 -- 172.21.15.32:0/4047992824 >> 172.21.15.32:0/4047992824 conn(0x7fd39006b930 msgr2=0x7fd39006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 -- 172.21.15.32:0/4047992824 shutdown_connections 2024-04-12T19:50:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.896+0000 7fd38f200640 1 -- 172.21.15.32:0/4047992824 wait complete. 2024-04-12T19:50:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.898+0000 7fd38f200640 1 Processor -- start 2024-04-12T19:50:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.898+0000 7fd38f200640 1 -- start start 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3900cee00 0x7fd39013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd39015cbe0 0x7fd39013ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd39006e3a0 con 0x7fd390071f50 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd39006e0c0 con 0x7fd3900cee00 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd39015d5b0 con 0x7fd39015cbe0 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60828/0 (socket says 172.21.15.32:60828) 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.899+0000 7fd38de00640 1 -- 172.21.15.32:0/99483747 learned_addr learned my addr 172.21.15.32:0/99483747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 -- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd39015cbe0 msgr2=0x7fd39013ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd39015cbe0 0x7fd39013ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 -- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3900cee00 msgr2=0x7fd39013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3900cee00 0x7fd39013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 -- 172.21.15.32:0/99483747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901586d0 con 0x7fd390071f50 2024-04-12T19:50:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38de00640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd380000c00 tx=0x7fd3800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380040070 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd380039bf0 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.900+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901589b0 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.901+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380035040 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.901+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd390158f70 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.902+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900b3f20 con 0x7fd390071f50 2024-04-12T19:50:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.903+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd38004b060 con 0x7fd390071f50 2024-04-12T19:50:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.903+0000 7fd386000640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.904+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd3800ad640 con 0x7fd390071f50 2024-04-12T19:50:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.904+0000 7fd387e00640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.904+0000 7fd387e00640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 0x7fd36c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd390156950 tx=0x7fd378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:24.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:24.908+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3900b3f20 con 0x7fd390071f50 2024-04-12T19:50:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.103+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3900755c0 con 0x7fd390071f50 2024-04-12T19:50:25.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.105+0000 7fd386000640 1 -- 172.21.15.32:0/99483747 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16610 (secure 0 0 0) 0x7fd380031610 con 0x7fd390071f50 2024-04-12T19:50:25.106 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:25.106 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:24.149654+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 msgr2=0x7fd36c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 0x7fd36c06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd390156950 tx=0x7fd378008040 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 msgr2=0x7fd390159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd380000c00 tx=0x7fd3800046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 shutdown_connections 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd39015cbe0 0x7fd39013ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3900cee00 0x7fd39013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 --2- 172.21.15.32:0/99483747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd390071f50 0x7fd390159770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.107+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 >> 172.21.15.32:0/99483747 conn(0x7fd39006b930 msgr2=0x7fd3900b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.108+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 shutdown_connections 2024-04-12T19:50:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.108+0000 7fd38f200640 1 -- 172.21.15.32:0/99483747 wait complete. 2024-04-12T19:50:25.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:25.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.341+0000 7f2827400640 1 Processor -- start 2024-04-12T19:50:25.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.342+0000 7f2827400640 1 -- start start 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.343+0000 7f2827400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f28201461f0 0x7f28200ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.343+0000 7f2827400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28200b3fe0 0x7f282015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.343+0000 7f2827400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28200b4be0 con 0x7f28200cedf0 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.343+0000 7f2827400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28200a9640 con 0x7f28200b3fe0 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.343+0000 7f2827400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f28200cf250 con 0x7f28201461f0 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28200b3fe0 0x7f282015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28200b3fe0 0x7f282015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60834/0 (socket says 172.21.15.32:60834) 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 -- 172.21.15.32:0/459334044 learned_addr learned my addr 172.21.15.32:0/459334044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:25.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2824a00640 1 --2- 172.21.15.32:0/459334044 >> v2:172.21.15.169:3300/0 conn(0x7f28201461f0 0x7f28200ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 -- 172.21.15.32:0/459334044 >> v2:172.21.15.169:3300/0 conn(0x7f28201461f0 msgr2=0x7f28200ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 --2- 172.21.15.32:0/459334044 >> v2:172.21.15.169:3300/0 conn(0x7f28201461f0 0x7f28200ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 -- 172.21.15.32:0/459334044 >> v1:172.21.15.169:6789/0 conn(0x7f28200cedf0 legacy=0x7f28200b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.344+0000 7f2825400640 1 -- 172.21.15.32:0/459334044 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28200b4620 con 0x7f28200b3fe0 2024-04-12T19:50:25.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.346+0000 7f2825400640 1 --2- 172.21.15.32:0/459334044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282015d7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f282006cf00 tx=0x7f281802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe675e426f3ede server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.346+0000 7f281f400640 1 -- 172.21.15.32:0/459334044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281802ca80 con 0x7f28200b3fe0 2024-04-12T19:50:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.346+0000 7f281f400640 1 -- 172.21.15.32:0/459334044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281802cc20 con 0x7f28200b3fe0 2024-04-12T19:50:25.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.346+0000 7f281f400640 1 -- 172.21.15.32:0/459334044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28180343f0 con 0x7f28200b3fe0 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 -- 172.21.15.32:0/459334044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 msgr2=0x7f282015d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 --2- 172.21.15.32:0/459334044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282015d7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f282006cf00 tx=0x7f281802bee0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 -- 172.21.15.32:0/459334044 shutdown_connections 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 --2- 172.21.15.32:0/459334044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282015d7b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 --2- 172.21.15.32:0/459334044 >> v2:172.21.15.169:3300/0 conn(0x7f28201461f0 0x7f28200ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 -- 172.21.15.32:0/459334044 >> 172.21.15.32:0/459334044 conn(0x7f28200b27b0 msgr2=0x7f28200b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:25.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 -- 172.21.15.32:0/459334044 shutdown_connections 2024-04-12T19:50:25.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.347+0000 7f2827400640 1 -- 172.21.15.32:0/459334044 wait complete. 2024-04-12T19:50:25.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.349+0000 7f2827400640 1 Processor -- start 2024-04-12T19:50:25.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.349+0000 7f2827400640 1 -- start start 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200cedf0 0x7f282013baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28201461f0 0x7f282013d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f282015d290 con 0x7f28200b3fe0 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f282015cfb0 con 0x7f28200cedf0 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2827400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28201465d0 con 0x7f28201461f0 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2824a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2824a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60840/0 (socket says 172.21.15.32:60840) 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.350+0000 7f2824a00640 1 -- 172.21.15.32:0/1098050637 learned_addr learned my addr 172.21.15.32:0/1098050637 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 -- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28201461f0 msgr2=0x7f282013d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28201461f0 0x7f282013d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f281fe00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200cedf0 0x7f282013baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 -- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200cedf0 msgr2=0x7f282013baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200cedf0 0x7f282013baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 -- 172.21.15.32:0/1098050637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282013bfe0 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f2824a00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f281000d5f0 tx=0x7f281000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100125f0 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2810019070 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.351+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100129d0 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.352+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282013fd20 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.352+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201402e0 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.353+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200cf9f0 con 0x7f28200b3fe0 2024-04-12T19:50:25.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.354+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2810012b70 con 0x7f28200b3fe0 2024-04-12T19:50:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.355+0000 7f281cc00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.355+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f281008bf10 con 0x7f28200b3fe0 2024-04-12T19:50:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.355+0000 7f281fe00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.356+0000 7f281fe00640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 0x7f280006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f282015c970 tx=0x7f280c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.359+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28200cf9f0 con 0x7f28200b3fe0 2024-04-12T19:50:25.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.564+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2820068c80 con 0x7f28200b3fe0 2024-04-12T19:50:25.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.566+0000 7f281cc00640 1 -- 172.21.15.32:0/1098050637 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16610 (secure 0 0 0) 0x7f281005a3d0 con 0x7f28200b3fe0 2024-04-12T19:50:25.567 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:25.567 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:24.149654+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.568+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 msgr2=0x7f280006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.568+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 0x7f280006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f282015c970 tx=0x7f280c009040 comp rx=0 tx=0).stop 2024-04-12T19:50:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 msgr2=0x7f282013f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f281000d5f0 tx=0x7f281000dab0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 shutdown_connections 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f280006c8f0 0x7f280006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28201461f0 0x7f282013d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200cedf0 0x7f282013baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 --2- 172.21.15.32:0/1098050637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28200b3fe0 0x7f282013f620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 >> 172.21.15.32:0/1098050637 conn(0x7f28200b27b0 msgr2=0x7f2820141780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 shutdown_connections 2024-04-12T19:50:25.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.569+0000 7f2827400640 1 -- 172.21.15.32:0/1098050637 wait complete. 2024-04-12T19:50:25.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-12T19:50:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.811+0000 7f16e5800640 1 Processor -- start 2024-04-12T19:50:25.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.812+0000 7f16e5800640 1 -- start start 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.813+0000 7f16e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.813+0000 7f16e5800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16e00d2b50 0x7f16e00d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.813+0000 7f16e5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e0162dd0 con 0x7f16e015c680 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.813+0000 7f16e5800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16e00d34f0 con 0x7f16e00d2b50 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.813+0000 7f16e5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16e0162b90 con 0x7f16e0162730 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16e00d2b50 0x7f16e00d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16e00d2b50 0x7f16e00d2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60856/0 (socket says 172.21.15.32:60856) 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 -- 172.21.15.32:0/1214234088 learned_addr learned my addr 172.21.15.32:0/1214234088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:25.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16df400640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 -- 172.21.15.32:0/1214234088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 msgr2=0x7f16e00b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 -- 172.21.15.32:0/1214234088 >> v1:172.21.15.169:6789/0 conn(0x7f16e015c680 legacy=0x7f16e015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.814+0000 7f16e4e00640 1 -- 172.21.15.32:0/1214234088 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e01630b0 con 0x7f16e00d2b50 2024-04-12T19:50:25.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.815+0000 7f16e4e00640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e00d2fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009870 tx=0x7f16d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848cbecfd655a9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16dea00640 1 -- 172.21.15.32:0/1214234088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8034070 con 0x7f16e00d2b50 2024-04-12T19:50:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16dea00640 1 -- 172.21.15.32:0/1214234088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d8030840 con 0x7f16e00d2b50 2024-04-12T19:50:25.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16dea00640 1 -- 172.21.15.32:0/1214234088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8030c10 con 0x7f16e00d2b50 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 -- 172.21.15.32:0/1214234088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 msgr2=0x7f16e00d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e00d2fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009870 tx=0x7f16d802b980 comp rx=0 tx=0).stop 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 -- 172.21.15.32:0/1214234088 shutdown_connections 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e00d2fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 --2- 172.21.15.32:0/1214234088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.816+0000 7f16e5800640 1 -- 172.21.15.32:0/1214234088 >> 172.21.15.32:0/1214234088 conn(0x7f16e0073480 msgr2=0x7f16e0073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.817+0000 7f16e5800640 1 -- 172.21.15.32:0/1214234088 shutdown_connections 2024-04-12T19:50:25.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.817+0000 7f16e5800640 1 -- 172.21.15.32:0/1214234088 wait complete. 2024-04-12T19:50:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.818+0000 7f16e5800640 1 Processor -- start 2024-04-12T19:50:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.819+0000 7f16e5800640 1 -- start start 2024-04-12T19:50:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 0x7f16e00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16e015ca60 con 0x7f16e00d2b50 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16e015f330 con 0x7f16e015c680 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16e015ff60 con 0x7f16e0162730 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60868/0 (socket says 172.21.15.32:60868) 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 -- 172.21.15.32:0/2183167024 learned_addr learned my addr 172.21.15.32:0/2183167024 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16e4e00640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16df400640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 0x7f16e00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 -- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 msgr2=0x7f16e00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 -- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 msgr2=0x7f16e00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 0x7f16e00c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16dfe00640 1 -- 172.21.15.32:0/2183167024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e00c9a50 con 0x7f16e00d2b50 2024-04-12T19:50:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.820+0000 7f16df400640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 0x7f16e00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16dfe00640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c430 tx=0x7f16d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00052d0 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d0005490 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0007260 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e01e8550 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.821+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01e8b10 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.822+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e00755c0 con 0x7f16e00d2b50 2024-04-12T19:50:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.823+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16d0007400 con 0x7f16e00d2b50 2024-04-12T19:50:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.824+0000 7f16d7e00640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 0x7f16bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.824+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f16d008b060 con 0x7f16e00d2b50 2024-04-12T19:50:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.825+0000 7f16df400640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 0x7f16bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.825+0000 7f16df400640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 0x7f16bc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f16e01f80d0 tx=0x7f16c800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:25.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:25.829+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16d0059490 con 0x7f16e00d2b50 2024-04-12T19:50:26.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.075+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f16e00b3460 con 0x7f16e00d2b50 2024-04-12T19:50:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.681+0000 7f16d7e00640 1 -- 172.21.15.32:0/2183167024 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f16d0058d90 con 0x7f16e00d2b50 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 msgr2=0x7f16bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 0x7f16bc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f16e01f80d0 tx=0x7f16c800b040 comp rx=0 tx=0).stop 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 msgr2=0x7f16e0149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c430 tx=0x7f16d000c8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 shutdown_connections 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f16bc06c8f0 0x7f16bc06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16e0162730 0x7f16e00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16e015c680 0x7f16e00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 --2- 172.21.15.32:0/2183167024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e00d2b50 0x7f16e0149c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.684+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 >> 172.21.15.32:0/2183167024 conn(0x7f16e0073480 msgr2=0x7f16e00d3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.685+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 shutdown_connections 2024-04-12T19:50:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.685+0000 7f16e5800640 1 -- 172.21.15.32:0/2183167024 wait complete. 2024-04-12T19:50:26.708 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-12T19:50:26.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-12T19:50:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.935+0000 7faa6f200640 1 Processor -- start 2024-04-12T19:50:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.935+0000 7faa6f200640 1 -- start start 2024-04-12T19:50:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa701478b0 0x7faa700afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6f200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa70147c90 0x7faa7006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6f200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa700aebd0 con 0x7faa7006f750 2024-04-12T19:50:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6f200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faa7006fbb0 con 0x7faa70147c90 2024-04-12T19:50:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faa700b0130 con 0x7faa701478b0 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa7006f750 0x7faa701626c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43190/0 (socket says 172.21.15.32:43190) 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6e800640 1 -- 172.21.15.32:0/3092660789 learned_addr learned my addr 172.21.15.32:0/3092660789 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa67e00640 1 --2- 172.21.15.32:0/3092660789 >> v2:172.21.15.32:3300/0 conn(0x7faa70147c90 0x7faa7006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.937+0000 7faa6de00640 1 --2- 172.21.15.32:0/3092660789 >> v2:172.21.15.169:3300/0 conn(0x7faa701478b0 0x7faa700afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1097201490 0 0) 0x7faa68001490 con 0x7faa7006f750 2024-04-12T19:50:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa50003520 con 0x7faa7006f750 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa67e00640 1 -- 172.21.15.32:0/3092660789 >> v2:172.21.15.169:3300/0 conn(0x7faa701478b0 msgr2=0x7faa700afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa67e00640 1 --2- 172.21.15.32:0/3092660789 >> v2:172.21.15.169:3300/0 conn(0x7faa701478b0 0x7faa700afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2998413449 0 0) 0x7faa50003520 con 0x7faa7006f750 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa67e00640 1 -- 172.21.15.32:0/3092660789 >> v1:172.21.15.32:6789/0 conn(0x7faa7006f750 legacy=0x7faa701626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.938+0000 7faa67e00640 1 -- 172.21.15.32:0/3092660789 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa700aee50 con 0x7faa70147c90 2024-04-12T19:50:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.939+0000 7faa67e00640 1 --2- 172.21.15.32:0/3092660789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa70147c90 0x7faa7006f210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa700b61c0 tx=0x7faa5802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11498de1a307c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58038070 con 0x7faa70147c90 2024-04-12T19:50:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa58033440 con 0x7faa70147c90 2024-04-12T19:50:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6d400640 1 -- 172.21.15.32:0/3092660789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68001bc0 con 0x7faa70147c90 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6f200640 1 -- 172.21.15.32:0/3092660789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa70147c90 msgr2=0x7faa7006f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6f200640 1 --2- 172.21.15.32:0/3092660789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa70147c90 0x7faa7006f210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa700b61c0 tx=0x7faa5802b7a0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.940+0000 7faa6f200640 1 -- 172.21.15.32:0/3092660789 shutdown_connections 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.941+0000 7faa6f200640 1 --2- 172.21.15.32:0/3092660789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa70147c90 0x7faa7006f210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.941+0000 7faa6f200640 1 --2- 172.21.15.32:0/3092660789 >> v2:172.21.15.169:3300/0 conn(0x7faa701478b0 0x7faa700afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.941+0000 7faa6f200640 1 -- 172.21.15.32:0/3092660789 >> 172.21.15.32:0/3092660789 conn(0x7faa7006b930 msgr2=0x7faa7006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.941+0000 7faa6f200640 1 -- 172.21.15.32:0/3092660789 shutdown_connections 2024-04-12T19:50:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.941+0000 7faa6f200640 1 -- 172.21.15.32:0/3092660789 wait complete. 2024-04-12T19:50:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.942+0000 7faa6f200640 1 Processor -- start 2024-04-12T19:50:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.943+0000 7faa6f200640 1 -- start start 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.943+0000 7faa6f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.943+0000 7faa6f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa701478b0 0x7faa700cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa70147c90 0x7faa700cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa700af6d0 con 0x7faa7006f750 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa700af3f0 con 0x7faa701478b0 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa70162dd0 con 0x7faa70147c90 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60888/0 (socket says 172.21.15.32:60888) 2024-04-12T19:50:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 -- 172.21.15.32:0/3533630025 learned_addr learned my addr 172.21.15.32:0/3533630025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 -- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa70147c90 msgr2=0x7faa700cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa70147c90 0x7faa700cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 -- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa701478b0 msgr2=0x7faa700cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa701478b0 0x7faa700cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 -- 172.21.15.32:0/3533630025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa700ccdb0 con 0x7faa7006f750 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.944+0000 7faa6de00640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa6000e9b0 tx=0x7faa6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.945+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60017070 con 0x7faa7006f750 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.945+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6000c9a0 con 0x7faa7006f750 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.945+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa700cd090 con 0x7faa7006f750 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.945+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6000cd90 con 0x7faa7006f750 2024-04-12T19:50:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.945+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa700cd650 con 0x7faa7006f750 2024-04-12T19:50:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.946+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7006c8c0 con 0x7faa7006f750 2024-04-12T19:50:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.947+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa6001a070 con 0x7faa7006f750 2024-04-12T19:50:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.948+0000 7faa66000640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 0x7faa4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.948+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faa6008bf60 con 0x7faa7006f750 2024-04-12T19:50:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.949+0000 7faa67e00640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 0x7faa4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.949+0000 7faa67e00640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 0x7faa4c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa58035000 tx=0x7faa5802d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:26.953+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa6005a380 con 0x7faa7006f750 2024-04-12T19:50:27.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.196+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7faa70072910 con 0x7faa7006f750 2024-04-12T19:50:27.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.692+0000 7faa66000640 1 -- 172.21.15.32:0/3533630025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7faa60059c80 con 0x7faa7006f750 2024-04-12T19:50:27.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 msgr2=0x7faa4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 0x7faa4c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa58035000 tx=0x7faa5802d000 comp rx=0 tx=0).stop 2024-04-12T19:50:27.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 msgr2=0x7faa7015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa6000e9b0 tx=0x7faa6000ee70 comp rx=0 tx=0).stop 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 shutdown_connections 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faa4c06c8f0 0x7faa4c06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa70147c90 0x7faa700cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa701478b0 0x7faa700cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 --2- 172.21.15.32:0/3533630025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7006f750 0x7faa7015db80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 >> 172.21.15.32:0/3533630025 conn(0x7faa7006b930 msgr2=0x7faa700b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 shutdown_connections 2024-04-12T19:50:27.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.696+0000 7faa6f200640 1 -- 172.21.15.32:0/3533630025 wait complete. 2024-04-12T19:50:27.721 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-12T19:50:27.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:50:27.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.946+0000 7fe948a00640 1 Processor -- start 2024-04-12T19:50:27.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.946+0000 7fe948a00640 1 -- start start 2024-04-12T19:50:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.947+0000 7fe948a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 0x7fe944162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.947+0000 7fe948a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.947+0000 7fe948a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9440d0090 con 0x7fe9440cf110 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.947+0000 7fe948a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe944163440 con 0x7fe9440d1c90 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.947+0000 7fe948a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9440b7b30 con 0x7fe944162020 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.948+0000 7fe943e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.948+0000 7fe943400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 0x7fe944162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.948+0000 7fe943e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51446/0 (socket says 172.21.15.32:51446) 2024-04-12T19:50:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.948+0000 7fe943e00640 1 -- 172.21.15.32:0/3711038148 learned_addr learned my addr 172.21.15.32:0/3711038148 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.949+0000 7fe943400640 1 -- 172.21.15.32:0/3711038148 >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 msgr2=0x7fe9440d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.949+0000 7fe943400640 1 --2- 172.21.15.32:0/3711038148 >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.949+0000 7fe943400640 1 -- 172.21.15.32:0/3711038148 >> v1:172.21.15.169:6789/0 conn(0x7fe9440cf110 legacy=0x7fe9440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.949+0000 7fe943400640 1 -- 172.21.15.32:0/3711038148 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9440cfad0 con 0x7fe944162020 2024-04-12T19:50:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.949+0000 7fe943e00640 1 --2- 172.21.15.32:0/3711038148 >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.950+0000 7fe943400640 1 --2- 172.21.15.32:0/3711038148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 0x7fe944162f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe9340097a0 tx=0x7fe93402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53bd0a97f13d496 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.951+0000 7fe942a00640 1 -- 172.21.15.32:0/3711038148 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934035020 con 0x7fe944162020 2024-04-12T19:50:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.951+0000 7fe942a00640 1 -- 172.21.15.32:0/3711038148 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934031810 con 0x7fe944162020 2024-04-12T19:50:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.951+0000 7fe942a00640 1 -- 172.21.15.32:0/3711038148 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934031be0 con 0x7fe944162020 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.951+0000 7fe948a00640 1 -- 172.21.15.32:0/3711038148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 msgr2=0x7fe944162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.951+0000 7fe948a00640 1 --2- 172.21.15.32:0/3711038148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 0x7fe944162f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe9340097a0 tx=0x7fe93402b8b0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 -- 172.21.15.32:0/3711038148 shutdown_connections 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 --2- 172.21.15.32:0/3711038148 >> v2:172.21.15.169:3300/0 conn(0x7fe9440d1c90 0x7fe9440d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 --2- 172.21.15.32:0/3711038148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe944162020 0x7fe944162f00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 -- 172.21.15.32:0/3711038148 >> 172.21.15.32:0/3711038148 conn(0x7fe94406cf00 msgr2=0x7fe94406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 -- 172.21.15.32:0/3711038148 shutdown_connections 2024-04-12T19:50:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.952+0000 7fe948a00640 1 -- 172.21.15.32:0/3711038148 wait complete. 2024-04-12T19:50:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.953+0000 7fe948a00640 1 Processor -- start 2024-04-12T19:50:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.954+0000 7fe948a00640 1 -- start start 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.954+0000 7fe948a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9440cf110 0x7fe944150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe948a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9440d1c90 0x7fe9440c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe948a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe948a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440d2b90 con 0x7fe944162020 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe948a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440cf880 con 0x7fe9440d1c90 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe948a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9440d2e70 con 0x7fe9440cf110 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60890/0 (socket says 172.21.15.32:60890) 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 -- 172.21.15.32:0/1020437235 learned_addr learned my addr 172.21.15.32:0/1020437235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe93be00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9440d1c90 0x7fe9440c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943400640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9440cf110 0x7fe944150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 -- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9440cf110 msgr2=0x7fe944150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9440cf110 0x7fe944150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 -- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9440d1c90 msgr2=0x7fe9440c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.955+0000 7fe943e00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9440d1c90 0x7fe9440c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe943e00640 1 -- 172.21.15.32:0/1020437235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441e9c90 con 0x7fe944162020 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe943e00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00efc0 tx=0x7fe93c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00cbe0 con 0x7fe944162020 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c00cd80 con 0x7fe944162020 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c003bb0 con 0x7fe944162020 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441e9f30 con 0x7fe944162020 2024-04-12T19:50:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.956+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441ea4f0 con 0x7fe944162020 2024-04-12T19:50:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.958+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9440755c0 con 0x7fe944162020 2024-04-12T19:50:27.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.958+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe93c01b030 con 0x7fe944162020 2024-04-12T19:50:27.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.959+0000 7fe93b400640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:27.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.959+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe93c08bf60 con 0x7fe944162020 2024-04-12T19:50:27.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.960+0000 7fe943400640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:27.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.960+0000 7fe943400640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 0x7fe92006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe934036000 tx=0x7fe934008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:27.964+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe93c022020 con 0x7fe944162020 2024-04-12T19:50:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.211+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9441ea6c0 con 0x7fe944162020 2024-04-12T19:50:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.213+0000 7fe93b400640 1 -- 172.21.15.32:0/1020437235 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7fe93c010040 con 0x7fe944162020 2024-04-12T19:50:28.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:28.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:50:28.214 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.215+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 msgr2=0x7fe92006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.215+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 0x7fe92006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe934036000 tx=0x7fe934008030 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.215+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 msgr2=0x7fe9441eaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.215+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00efc0 tx=0x7fe93c00c490 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 shutdown_connections 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe944162020 0x7fe9441eaa10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9440d1c90 0x7fe9440c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 --2- 172.21.15.32:0/1020437235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9440cf110 0x7fe944150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 >> 172.21.15.32:0/1020437235 conn(0x7fe94406cf00 msgr2=0x7fe9440cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 shutdown_connections 2024-04-12T19:50:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.216+0000 7fe948a00640 1 -- 172.21.15.32:0/1020437235 wait complete. 2024-04-12T19:50:28.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.458+0000 7f25fa800640 1 Processor -- start 2024-04-12T19:50:28.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.458+0000 7f25fa800640 1 -- start start 2024-04-12T19:50:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.460+0000 7f25fa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40cee00 0x7f25f406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.460+0000 7f25fa800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25f4071f50 0x7f25f40723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.460+0000 7f25fa800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f415dbb0 con 0x7f25f415cbe0 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.460+0000 7f25fa800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25f406f4f0 con 0x7f25f4071f50 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.460+0000 7f25fa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40728f0 con 0x7f25f40cee00 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.461+0000 7f25f9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40cee00 0x7f25f406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.461+0000 7f25f8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25f4071f50 0x7f25f40723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.461+0000 7f25f8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25f4071f50 0x7f25f40723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51464/0 (socket says 172.21.15.32:51464) 2024-04-12T19:50:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.461+0000 7f25f8a00640 1 -- 172.21.15.32:0/2457242338 learned_addr learned my addr 172.21.15.32:0/2457242338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.462+0000 7f25f8a00640 1 -- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40cee00 msgr2=0x7f25f406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.462+0000 7f25f8a00640 1 --2- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40cee00 0x7f25f406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.462+0000 7f25f8a00640 1 -- 172.21.15.32:0/2457242338 >> v1:172.21.15.169:6789/0 conn(0x7f25f415cbe0 legacy=0x7f25f415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.462+0000 7f25f8a00640 1 -- 172.21.15.32:0/2457242338 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f415d8d0 con 0x7f25f4071f50 2024-04-12T19:50:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.463+0000 7f25f8a00640 1 --2- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f4071f50 0x7f25f40723b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25e00098a0 tx=0x7f25e002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ae1c4dc6d081e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25f3e00640 1 -- 172.21.15.32:0/2457242338 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0034070 con 0x7f25f4071f50 2024-04-12T19:50:28.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25f3e00640 1 -- 172.21.15.32:0/2457242338 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e0030840 con 0x7f25f4071f50 2024-04-12T19:50:28.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25f3e00640 1 -- 172.21.15.32:0/2457242338 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0030c10 con 0x7f25f4071f50 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 -- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f4071f50 msgr2=0x7f25f40723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 --2- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f4071f50 0x7f25f40723b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f25e00098a0 tx=0x7f25e002b980 comp rx=0 tx=0).stop 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 -- 172.21.15.32:0/2457242338 shutdown_connections 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 --2- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f4071f50 0x7f25f40723b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 --2- 172.21.15.32:0/2457242338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40cee00 0x7f25f406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 -- 172.21.15.32:0/2457242338 >> 172.21.15.32:0/2457242338 conn(0x7f25f406b930 msgr2=0x7f25f406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 -- 172.21.15.32:0/2457242338 shutdown_connections 2024-04-12T19:50:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.464+0000 7f25fa800640 1 -- 172.21.15.32:0/2457242338 wait complete. 2024-04-12T19:50:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.466+0000 7f25fa800640 1 Processor -- start 2024-04-12T19:50:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.466+0000 7f25fa800640 1 -- start start 2024-04-12T19:50:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 0x7f25f4160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 0x7f25f4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25f406e3a0 con 0x7f25f4071f50 2024-04-12T19:50:28.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25f406e0c0 con 0x7f25f40cee00 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.467+0000 7f25fa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25f415dda0 con 0x7f25f415cbe0 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60892/0 (socket says 172.21.15.32:60892) 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 -- 172.21.15.32:0/390169674 learned_addr learned my addr 172.21.15.32:0/390169674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f8a00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 0x7f25f4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9e00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 0x7f25f4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 -- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 msgr2=0x7f25f4155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 0x7f25f4155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 -- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 msgr2=0x7f25f4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 0x7f25f4160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9400640 1 -- 172.21.15.32:0/390169674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f415f2e0 con 0x7f25f4071f50 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f8a00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 0x7f25f4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.468+0000 7f25f9e00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 0x7f25f4155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25f9400640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009db0 tx=0x7f25e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8018020 con 0x7f25f4071f50 2024-04-12T19:50:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8014480 con 0x7f25f4071f50 2024-04-12T19:50:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f415f5c0 con 0x7f25f4071f50 2024-04-12T19:50:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8014850 con 0x7f25f4071f50 2024-04-12T19:50:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.469+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f415fb80 con 0x7f25f4071f50 2024-04-12T19:50:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.470+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f40755c0 con 0x7f25f4071f50 2024-04-12T19:50:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.471+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25e8005d10 con 0x7f25f4071f50 2024-04-12T19:50:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.472+0000 7f25f1600640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 0x7f25d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.472+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25e808c6b0 con 0x7f25f4071f50 2024-04-12T19:50:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.472+0000 7f25f8a00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 0x7f25d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.473+0000 7f25f8a00640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 0x7f25d406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25e0036000 tx=0x7f25e003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.477+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25e8022070 con 0x7f25f4071f50 2024-04-12T19:50:28.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.681+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25f415c010 con 0x7f25f4071f50 2024-04-12T19:50:28.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.684+0000 7f25f1600640 1 -- 172.21.15.32:0/390169674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7f25e8010040 con 0x7f25f4071f50 2024-04-12T19:50:28.685 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:28.685 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 msgr2=0x7f25d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 0x7f25d406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25e0036000 tx=0x7f25e003b040 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 msgr2=0x7f25f414a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009db0 tx=0x7f25e800c490 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 shutdown_connections 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25d406c920 0x7f25d406ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f415cbe0 0x7f25f4155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40cee00 0x7f25f4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 --2- 172.21.15.32:0/390169674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25f4071f50 0x7f25f414a170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 >> 172.21.15.32:0/390169674 conn(0x7f25f406b930 msgr2=0x7f25f40cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.687+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 shutdown_connections 2024-04-12T19:50:28.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.688+0000 7f25fa800640 1 -- 172.21.15.32:0/390169674 wait complete. 2024-04-12T19:50:28.710 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-12T19:50:28.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:50:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.930+0000 7f6ebaa00640 1 Processor -- start 2024-04-12T19:50:28.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.930+0000 7f6ebaa00640 1 -- start start 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6eb4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6ebaa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb40d0090 con 0x7f6eb40cf110 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6ebaa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb40cfad0 con 0x7f6eb4162020 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6ebaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb4163440 con 0x7f6eb40d1c90 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6eba000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6eb40cf110 0x7f6eb40d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43200/0 (socket says 172.21.15.32:43200) 2024-04-12T19:50:28.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.932+0000 7f6eba000640 1 -- 172.21.15.32:0/1725815193 learned_addr learned my addr 172.21.15.32:0/1725815193 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6eb4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579841056 0 0) 0x7f6eac001490 con 0x7f6eb40cf110 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e9c003520 con 0x7f6eb40cf110 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707727242 0 0) 0x7f6e9c003520 con 0x7f6eb40cf110 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 msgr2=0x7f6eb4162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6eb4162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 >> v1:172.21.15.169:6789/0 conn(0x7f6eb4162020 legacy=0x7f6eb40d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb40cfdb0 con 0x7f6eb40cf110 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6eac001bc0 con 0x7f6eb40cf110 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 >> v1:172.21.15.32:6789/0 conn(0x7f6eb40cf110 legacy=0x7f6eb40d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9c0039e0 0x7f6e9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9c004800 0x7f6e9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6e9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4163440 con 0x7f6e9c004800 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c00baf0 con 0x7f6e9c0039e0 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.933+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9c00bcf0 con 0x7f6eb40d1c90 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9c004800 0x7f6e9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb9600640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9c0039e0 0x7f6e9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eba000640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6e9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 -- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 msgr2=0x7f6e9c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb40d1c90 0x7f6e9c005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 -- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9c0039e0 msgr2=0x7f6e9c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9c0039e0 0x7f6e9c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb8c00640 1 -- 172.21.15.32:0/1725815193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c003520 con 0x7f6e9c004800 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.934+0000 7f6eb9600640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e9c0039e0 0x7f6e9c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:28.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.935+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9c004800 0x7f6e9c004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000ba00 tx=0x7f6e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7b9b5e463dab4e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.935+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0036020 con 0x7f6e9c004800 2024-04-12T19:50:28.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.935+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0032620 con 0x7f6e9c004800 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.935+0000 7f6eb3e00640 1 -- 172.21.15.32:0/1725815193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000b600 con 0x7f6e9c004800 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 -- 172.21.15.32:0/1725815193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9c004800 msgr2=0x7f6e9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/1725815193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9c004800 0x7f6e9c004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000ba00 tx=0x7f6e9c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6eba000640 1 -- 172.21.15.32:0/1725815193 reap_dead start 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 -- 172.21.15.32:0/1725815193 shutdown_connections 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 -- 172.21.15.32:0/1725815193 >> 172.21.15.32:0/1725815193 conn(0x7f6eb406cf00 msgr2=0x7f6eb406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 -- 172.21.15.32:0/1725815193 shutdown_connections 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.936+0000 7f6ebaa00640 1 -- 172.21.15.32:0/1725815193 wait complete. 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.938+0000 7f6ebaa00640 1 Processor -- start 2024-04-12T19:50:28.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.938+0000 7f6ebaa00640 1 -- start start 2024-04-12T19:50:28.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e9c004800 0x7f6eb4160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 0x7f6eb415f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb40cf880 con 0x7f6eb40cafb0 2024-04-12T19:50:28.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb40b7b30 con 0x7f6eb40cab30 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6ebaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb40d2e70 con 0x7f6e9c004800 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6eba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6eba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60910/0 (socket says 172.21.15.32:60910) 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.939+0000 7f6eba000640 1 -- 172.21.15.32:0/3579020300 learned_addr learned my addr 172.21.15.32:0/3579020300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 0x7f6eb415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eb9600640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e9c004800 0x7f6eb4160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 -- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e9c004800 msgr2=0x7f6eb4160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e9c004800 0x7f6eb4160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 -- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 msgr2=0x7f6eb415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 0x7f6eb415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 -- 172.21.15.32:0/3579020300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb415e080 con 0x7f6eb40cafb0 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eb8c00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 0x7f6eb415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eba000640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40ba230 tx=0x7f6eac00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.940+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004d60 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.941+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eac00bb80 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.941+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac007070 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.941+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb415e360 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.941+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb415e920 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.942+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb40755c0 con 0x7f6eb40cafb0 2024-04-12T19:50:28.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.943+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6eac01b020 con 0x7f6eb40cafb0 2024-04-12T19:50:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.944+0000 7f6eb1600640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 0x7f6e9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.944+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6eac08bd10 con 0x7f6eb40cafb0 2024-04-12T19:50:28.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.944+0000 7f6eb9600640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 0x7f6e9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:28.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.945+0000 7f6eb9600640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 0x7f6e9406ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8001750 tx=0x7f6ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:28.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:28.949+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6eac091020 con 0x7f6eb40cafb0 2024-04-12T19:50:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.150+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6eb40b7650 con 0x7f6e9406c920 2024-04-12T19:50:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.161+0000 7f6eb1600640 1 -- 172.21.15.32:0/3579020300 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f6eb40b7650 con 0x7f6e9406c920 2024-04-12T19:50:29.165 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:50:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 msgr2=0x7f6e9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 0x7f6e9406ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8001750 tx=0x7f6ea8009040 comp rx=0 tx=0).stop 2024-04-12T19:50:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 msgr2=0x7f6eb40cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40ba230 tx=0x7f6eac00b4a0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 shutdown_connections 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e9406c920 0x7f6e9406ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eb40cafb0 0x7f6eb40cd900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb40cab30 0x7f6eb415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 --2- 172.21.15.32:0/3579020300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e9c004800 0x7f6eb4160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 >> 172.21.15.32:0/3579020300 conn(0x7f6eb406cf00 msgr2=0x7f6eb40b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 shutdown_connections 2024-04-12T19:50:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.167+0000 7f6ebaa00640 1 -- 172.21.15.32:0/3579020300 wait complete. 2024-04-12T19:50:29.186 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-12T19:50:27.625738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235744,"kb_used_data":2928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738976,"statfs":{"total":773094113280,"available":772852711424,"internally_reserved":0,"allocated":2998272,"data_stored":1783228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992161"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689002+0000","last_change":"2024-04-12T19:50:24.687884+0000","last_active":"2024-04-12T19:50:25.689002+0000","last_peered":"2024-04-12T19:50:25.689002+0000","last_clean":"2024-04-12T19:50:25.689002+0000","last_became_active":"2024-04-12T19:50:24.686545+0000","last_became_peered":"2024-04-12T19:50:24.686545+0000","last_unstale":"2024-04-12T19:50:25.689002+0000","last_undegraded":"2024-04-12T19:50:25.689002+0000","last_fullsized":"2024-04-12T19:50:25.689002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194273+0000","last_change":"2024-04-12T19:50:24.693720+0000","last_active":"2024-04-12T19:50:26.194273+0000","last_peered":"2024-04-12T19:50:26.194273+0000","last_clean":"2024-04-12T19:50:26.194273+0000","last_became_active":"2024-04-12T19:50:24.693332+0000","last_became_peered":"2024-04-12T19:50:24.693332+0000","last_unstale":"2024-04-12T19:50:26.194273+0000","last_undegraded":"2024-04-12T19:50:26.194273+0000","last_fullsized":"2024-04-12T19:50:26.194273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194432+0000","last_change":"2024-04-12T19:50:24.688591+0000","last_active":"2024-04-12T19:50:26.194432+0000","last_peered":"2024-04-12T19:50:26.194432+0000","last_clean":"2024-04-12T19:50:26.194432+0000","last_became_active":"2024-04-12T19:50:24.688345+0000","last_became_peered":"2024-04-12T19:50:24.688345+0000","last_unstale":"2024-04-12T19:50:26.194432+0000","last_undegraded":"2024-04-12T19:50:26.194432+0000","last_fullsized":"2024-04-12T19:50:26.194432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675845+0000","last_change":"2024-04-12T19:50:24.680503+0000","last_active":"2024-04-12T19:50:25.675845+0000","last_peered":"2024-04-12T19:50:25.675845+0000","last_clean":"2024-04-12T19:50:25.675845+0000","last_became_active":"2024-04-12T19:50:24.674820+0000","last_became_peered":"2024-04-12T19:50:24.674820+0000","last_unstale":"2024-04-12T19:50:25.675845+0000","last_undegraded":"2024-04-12T19:50:25.675845+0000","last_fullsized":"2024-04-12T19:50:25.675845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688640+0000","last_change":"2024-04-12T19:50:24.688597+0000","last_active":"2024-04-12T19:50:25.688640+0000","last_peered":"2024-04-12T19:50:25.688640+0000","last_clean":"2024-04-12T19:50:25.688640+0000","last_became_active":"2024-04-12T19:50:24.688308+0000","last_became_peered":"2024-04-12T19:50:24.688308+0000","last_unstale":"2024-04-12T19:50:25.688640+0000","last_undegraded":"2024-04-12T19:50:25.688640+0000","last_fullsized":"2024-04-12T19:50:25.688640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675161+0000","last_change":"2024-04-12T19:50:24.680554+0000","last_active":"2024-04-12T19:50:25.675161+0000","last_peered":"2024-04-12T19:50:25.675161+0000","last_clean":"2024-04-12T19:50:25.675161+0000","last_became_active":"2024-04-12T19:50:24.675178+0000","last_became_peered":"2024-04-12T19:50:24.675178+0000","last_unstale":"2024-04-12T19:50:25.675161+0000","last_undegraded":"2024-04-12T19:50:25.675161+0000","last_fullsized":"2024-04-12T19:50:25.675161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688158+0000","last_change":"2024-04-12T19:50:24.677924+0000","last_active":"2024-04-12T19:50:25.688158+0000","last_peered":"2024-04-12T19:50:25.688158+0000","last_clean":"2024-04-12T19:50:25.688158+0000","last_became_active":"2024-04-12T19:50:24.677414+0000","last_became_peered":"2024-04-12T19:50:24.677414+0000","last_unstale":"2024-04-12T19:50:25.688158+0000","last_undegraded":"2024-04-12T19:50:25.688158+0000","last_fullsized":"2024-04-12T19:50:25.688158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193170+0000","last_change":"2024-04-12T19:50:24.690479+0000","last_active":"2024-04-12T19:50:26.193170+0000","last_peered":"2024-04-12T19:50:26.193170+0000","last_clean":"2024-04-12T19:50:26.193170+0000","last_became_active":"2024-04-12T19:50:24.690010+0000","last_became_peered":"2024-04-12T19:50:24.690010+0000","last_unstale":"2024-04-12T19:50:26.193170+0000","last_undegraded":"2024-04-12T19:50:26.193170+0000","last_fullsized":"2024-04-12T19:50:26.193170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673588+0000","last_change":"2024-04-12T19:50:24.679542+0000","last_active":"2024-04-12T19:50:25.673588+0000","last_peered":"2024-04-12T19:50:25.673588+0000","last_clean":"2024-04-12T19:50:25.673588+0000","last_became_active":"2024-04-12T19:50:24.678975+0000","last_became_peered":"2024-04-12T19:50:24.678975+0000","last_unstale":"2024-04-12T19:50:25.673588+0000","last_undegraded":"2024-04-12T19:50:25.673588+0000","last_fullsized":"2024-04-12T19:50:25.673588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688977+0000","last_change":"2024-04-12T19:50:24.681572+0000","last_active":"2024-04-12T19:50:25.688977+0000","last_peered":"2024-04-12T19:50:25.688977+0000","last_clean":"2024-04-12T19:50:25.688977+0000","last_became_active":"2024-04-12T19:50:24.681309+0000","last_became_peered":"2024-04-12T19:50:24.681309+0000","last_unstale":"2024-04-12T19:50:25.688977+0000","last_undegraded":"2024-04-12T19:50:25.688977+0000","last_fullsized":"2024-04-12T19:50:25.688977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688257+0000","last_change":"2024-04-12T19:50:24.678439+0000","last_active":"2024-04-12T19:50:25.688257+0000","last_peered":"2024-04-12T19:50:25.688257+0000","last_clean":"2024-04-12T19:50:25.688257+0000","last_became_active":"2024-04-12T19:50:24.677934+0000","last_became_peered":"2024-04-12T19:50:24.677934+0000","last_unstale":"2024-04-12T19:50:25.688257+0000","last_undegraded":"2024-04-12T19:50:25.688257+0000","last_fullsized":"2024-04-12T19:50:25.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688579+0000","last_change":"2024-04-12T19:50:24.683054+0000","last_active":"2024-04-12T19:50:25.688579+0000","last_peered":"2024-04-12T19:50:25.688579+0000","last_clean":"2024-04-12T19:50:25.688579+0000","last_became_active":"2024-04-12T19:50:24.682749+0000","last_became_peered":"2024-04-12T19:50:24.682749+0000","last_unstale":"2024-04-12T19:50:25.688579+0000","last_undegraded":"2024-04-12T19:50:25.688579+0000","last_fullsized":"2024-04-12T19:50:25.688579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675969+0000","last_change":"2024-04-12T19:50:24.700812+0000","last_active":"2024-04-12T19:50:25.675969+0000","last_peered":"2024-04-12T19:50:25.675969+0000","last_clean":"2024-04-12T19:50:25.675969+0000","last_became_active":"2024-04-12T19:50:24.700470+0000","last_became_peered":"2024-04-12T19:50:24.700470+0000","last_unstale":"2024-04-12T19:50:25.675969+0000","last_undegraded":"2024-04-12T19:50:25.675969+0000","last_fullsized":"2024-04-12T19:50:25.675969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676868+0000","last_change":"2024-04-12T19:50:24.678150+0000","last_active":"2024-04-12T19:50:25.676868+0000","last_peered":"2024-04-12T19:50:25.676868+0000","last_clean":"2024-04-12T19:50:25.676868+0000","last_became_active":"2024-04-12T19:50:24.677811+0000","last_became_peered":"2024-04-12T19:50:24.677811+0000","last_unstale":"2024-04-12T19:50:25.676868+0000","last_undegraded":"2024-04-12T19:50:25.676868+0000","last_fullsized":"2024-04-12T19:50:25.676868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193225+0000","last_change":"2024-04-12T19:50:24.701334+0000","last_active":"2024-04-12T19:50:26.193225+0000","last_peered":"2024-04-12T19:50:26.193225+0000","last_clean":"2024-04-12T19:50:26.193225+0000","last_became_active":"2024-04-12T19:50:24.701046+0000","last_became_peered":"2024-04-12T19:50:24.701046+0000","last_unstale":"2024-04-12T19:50:26.193225+0000","last_undegraded":"2024-04-12T19:50:26.193225+0000","last_fullsized":"2024-04-12T19:50:26.193225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184164+0000","last_change":"2024-04-12T19:50:24.678787+0000","last_active":"2024-04-12T19:50:26.184164+0000","last_peered":"2024-04-12T19:50:26.184164+0000","last_clean":"2024-04-12T19:50:26.184164+0000","last_became_active":"2024-04-12T19:50:24.678537+0000","last_became_peered":"2024-04-12T19:50:24.678537+0000","last_unstale":"2024-04-12T19:50:26.184164+0000","last_undegraded":"2024-04-12T19:50:26.184164+0000","last_fullsized":"2024-04-12T19:50:26.184164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185746+0000","last_change":"2024-04-12T19:50:24.680249+0000","last_active":"2024-04-12T19:50:26.185746+0000","last_peered":"2024-04-12T19:50:26.185746+0000","last_clean":"2024-04-12T19:50:26.185746+0000","last_became_active":"2024-04-12T19:50:24.679835+0000","last_became_peered":"2024-04-12T19:50:24.679835+0000","last_unstale":"2024-04-12T19:50:26.185746+0000","last_undegraded":"2024-04-12T19:50:26.185746+0000","last_fullsized":"2024-04-12T19:50:26.185746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:17:07.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.699165+0000","last_change":"2024-04-12T19:50:24.697806+0000","last_active":"2024-04-12T19:50:24.699165+0000","last_peered":"2024-04-12T19:50:24.699165+0000","last_clean":"2024-04-12T19:50:24.699165+0000","last_became_active":"2024-04-12T19:50:24.697515+0000","last_became_peered":"2024-04-12T19:50:24.697515+0000","last_unstale":"2024-04-12T19:50:24.699165+0000","last_undegraded":"2024-04-12T19:50:24.699165+0000","last_fullsized":"2024-04-12T19:50:24.699165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:06:20.109649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185806+0000","last_change":"2024-04-12T19:50:24.684475+0000","last_active":"2024-04-12T19:50:26.185806+0000","last_peered":"2024-04-12T19:50:26.185806+0000","last_clean":"2024-04-12T19:50:26.185806+0000","last_became_active":"2024-04-12T19:50:24.684052+0000","last_became_peered":"2024-04-12T19:50:24.684052+0000","last_unstale":"2024-04-12T19:50:26.185806+0000","last_undegraded":"2024-04-12T19:50:26.185806+0000","last_fullsized":"2024-04-12T19:50:26.185806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:19:38.175136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688476+0000","last_change":"2024-04-12T19:50:24.679515+0000","last_active":"2024-04-12T19:50:25.688476+0000","last_peered":"2024-04-12T19:50:25.688476+0000","last_clean":"2024-04-12T19:50:25.688476+0000","last_became_active":"2024-04-12T19:50:24.678757+0000","last_became_peered":"2024-04-12T19:50:24.678757+0000","last_unstale":"2024-04-12T19:50:25.688476+0000","last_undegraded":"2024-04-12T19:50:25.688476+0000","last_fullsized":"2024-04-12T19:50:25.688476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682423+0000","last_change":"2024-04-12T19:50:24.674490+0000","last_active":"2024-04-12T19:50:25.682423+0000","last_peered":"2024-04-12T19:50:25.682423+0000","last_clean":"2024-04-12T19:50:25.682423+0000","last_became_active":"2024-04-12T19:50:24.674131+0000","last_became_peered":"2024-04-12T19:50:24.674131+0000","last_unstale":"2024-04-12T19:50:25.682423+0000","last_undegraded":"2024-04-12T19:50:25.682423+0000","last_fullsized":"2024-04-12T19:50:25.682423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673126+0000","last_change":"2024-04-12T19:50:24.673011+0000","last_active":"2024-04-12T19:50:25.673126+0000","last_peered":"2024-04-12T19:50:25.673126+0000","last_clean":"2024-04-12T19:50:25.673126+0000","last_became_active":"2024-04-12T19:50:24.672490+0000","last_became_peered":"2024-04-12T19:50:24.672490+0000","last_unstale":"2024-04-12T19:50:25.673126+0000","last_undegraded":"2024-04-12T19:50:25.673126+0000","last_fullsized":"2024-04-12T19:50:25.673126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193541+0000","last_change":"2024-04-12T19:50:24.692657+0000","last_active":"2024-04-12T19:50:26.193541+0000","last_peered":"2024-04-12T19:50:26.193541+0000","last_clean":"2024-04-12T19:50:26.193541+0000","last_became_active":"2024-04-12T19:50:24.692335+0000","last_became_peered":"2024-04-12T19:50:24.692335+0000","last_unstale":"2024-04-12T19:50:26.193541+0000","last_undegraded":"2024-04-12T19:50:26.193541+0000","last_fullsized":"2024-04-12T19:50:26.193541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184179+0000","last_change":"2024-04-12T19:50:24.692524+0000","last_active":"2024-04-12T19:50:26.184179+0000","last_peered":"2024-04-12T19:50:26.184179+0000","last_clean":"2024-04-12T19:50:26.184179+0000","last_became_active":"2024-04-12T19:50:24.692239+0000","last_became_peered":"2024-04-12T19:50:24.692239+0000","last_unstale":"2024-04-12T19:50:26.184179+0000","last_undegraded":"2024-04-12T19:50:26.184179+0000","last_fullsized":"2024-04-12T19:50:26.184179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193671+0000","last_change":"2024-04-12T19:50:24.686815+0000","last_active":"2024-04-12T19:50:26.193671+0000","last_peered":"2024-04-12T19:50:26.193671+0000","last_clean":"2024-04-12T19:50:26.193671+0000","last_became_active":"2024-04-12T19:50:24.686192+0000","last_became_peered":"2024-04-12T19:50:24.686192+0000","last_unstale":"2024-04-12T19:50:26.193671+0000","last_undegraded":"2024-04-12T19:50:26.193671+0000","last_fullsized":"2024-04-12T19:50:26.193671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676094+0000","last_change":"2024-04-12T19:50:24.686489+0000","last_active":"2024-04-12T19:50:25.676094+0000","last_peered":"2024-04-12T19:50:25.676094+0000","last_clean":"2024-04-12T19:50:25.676094+0000","last_became_active":"2024-04-12T19:50:24.685753+0000","last_became_peered":"2024-04-12T19:50:24.685753+0000","last_unstale":"2024-04-12T19:50:25.676094+0000","last_undegraded":"2024-04-12T19:50:25.676094+0000","last_fullsized":"2024-04-12T19:50:25.676094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676600+0000","last_change":"2024-04-12T19:50:24.675093+0000","last_active":"2024-04-12T19:50:25.676600+0000","last_peered":"2024-04-12T19:50:25.676600+0000","last_clean":"2024-04-12T19:50:25.676600+0000","last_became_active":"2024-04-12T19:50:24.674798+0000","last_became_peered":"2024-04-12T19:50:24.674798+0000","last_unstale":"2024-04-12T19:50:25.676600+0000","last_undegraded":"2024-04-12T19:50:25.676600+0000","last_fullsized":"2024-04-12T19:50:25.676600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676605+0000","last_change":"2024-04-12T19:50:24.681961+0000","last_active":"2024-04-12T19:50:25.676605+0000","last_peered":"2024-04-12T19:50:25.676605+0000","last_clean":"2024-04-12T19:50:25.676605+0000","last_became_active":"2024-04-12T19:50:24.681596+0000","last_became_peered":"2024-04-12T19:50:24.681596+0000","last_unstale":"2024-04-12T19:50:25.676605+0000","last_undegraded":"2024-04-12T19:50:25.676605+0000","last_fullsized":"2024-04-12T19:50:25.676605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194168+0000","last_change":"2024-04-12T19:50:24.688114+0000","last_active":"2024-04-12T19:50:26.194168+0000","last_peered":"2024-04-12T19:50:26.194168+0000","last_clean":"2024-04-12T19:50:26.194168+0000","last_became_active":"2024-04-12T19:50:24.687826+0000","last_became_peered":"2024-04-12T19:50:24.687826+0000","last_unstale":"2024-04-12T19:50:26.194168+0000","last_undegraded":"2024-04-12T19:50:26.194168+0000","last_fullsized":"2024-04-12T19:50:26.194168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184379+0000","last_change":"2024-04-12T19:50:24.697023+0000","last_active":"2024-04-12T19:50:26.184379+0000","last_peered":"2024-04-12T19:50:26.184379+0000","last_clean":"2024-04-12T19:50:26.184379+0000","last_became_active":"2024-04-12T19:50:24.696739+0000","last_became_peered":"2024-04-12T19:50:24.696739+0000","last_unstale":"2024-04-12T19:50:26.184379+0000","last_undegraded":"2024-04-12T19:50:26.184379+0000","last_fullsized":"2024-04-12T19:50:26.184379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:46:45.869946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.680237+0000","last_change":"2024-04-12T19:50:24.675081+0000","last_active":"2024-04-12T19:50:24.680237+0000","last_peered":"2024-04-12T19:50:24.680237+0000","last_clean":"2024-04-12T19:50:24.680237+0000","last_became_active":"2024-04-12T19:50:24.674812+0000","last_became_peered":"2024-04-12T19:50:24.674812+0000","last_unstale":"2024-04-12T19:50:24.680237+0000","last_undegraded":"2024-04-12T19:50:24.680237+0000","last_fullsized":"2024-04-12T19:50:24.680237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:02:45.126635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.674913+0000","last_change":"2024-04-12T19:50:24.687854+0000","last_active":"2024-04-12T19:50:25.674913+0000","last_peered":"2024-04-12T19:50:25.674913+0000","last_clean":"2024-04-12T19:50:25.674913+0000","last_became_active":"2024-04-12T19:50:24.685605+0000","last_became_peered":"2024-04-12T19:50:24.685605+0000","last_unstale":"2024-04-12T19:50:25.674913+0000","last_undegraded":"2024-04-12T19:50:25.674913+0000","last_fullsized":"2024-04-12T19:50:25.674913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:11.962955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673270+0000","last_change":"2024-04-12T19:50:24.694100+0000","last_active":"2024-04-12T19:50:25.673270+0000","last_peered":"2024-04-12T19:50:25.673270+0000","last_clean":"2024-04-12T19:50:25.673270+0000","last_became_active":"2024-04-12T19:50:24.693828+0000","last_became_peered":"2024-04-12T19:50:24.693828+0000","last_unstale":"2024-04-12T19:50:25.673270+0000","last_undegraded":"2024-04-12T19:50:25.673270+0000","last_fullsized":"2024-04-12T19:50:25.673270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194122+0000","last_change":"2024-04-12T19:50:23.685737+0000","last_active":"2024-04-12T19:50:26.194122+0000","last_peered":"2024-04-12T19:50:26.194122+0000","last_clean":"2024-04-12T19:50:26.194122+0000","last_became_active":"2024-04-12T19:50:23.685371+0000","last_became_peered":"2024-04-12T19:50:23.685371+0000","last_unstale":"2024-04-12T19:50:26.194122+0000","last_undegraded":"2024-04-12T19:50:26.194122+0000","last_fullsized":"2024-04-12T19:50:26.194122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682544+0000","last_change":"2024-04-12T19:50:24.681008+0000","last_active":"2024-04-12T19:50:25.682544+0000","last_peered":"2024-04-12T19:50:25.682544+0000","last_clean":"2024-04-12T19:50:25.682544+0000","last_became_active":"2024-04-12T19:50:24.680689+0000","last_became_peered":"2024-04-12T19:50:24.680689+0000","last_unstale":"2024-04-12T19:50:25.682544+0000","last_undegraded":"2024-04-12T19:50:25.682544+0000","last_fullsized":"2024-04-12T19:50:25.682544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184687+0000","last_change":"2024-04-12T19:50:23.691933+0000","last_active":"2024-04-12T19:50:26.184687+0000","last_peered":"2024-04-12T19:50:26.184687+0000","last_clean":"2024-04-12T19:50:26.184687+0000","last_became_active":"2024-04-12T19:50:23.691622+0000","last_became_peered":"2024-04-12T19:50:23.691622+0000","last_unstale":"2024-04-12T19:50:26.184687+0000","last_undegraded":"2024-04-12T19:50:26.184687+0000","last_fullsized":"2024-04-12T19:50:26.184687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675732+0000","last_change":"2024-04-12T19:50:23.691962+0000","last_active":"2024-04-12T19:50:25.675732+0000","last_peered":"2024-04-12T19:50:25.675732+0000","last_clean":"2024-04-12T19:50:25.675732+0000","last_became_active":"2024-04-12T19:50:23.691571+0000","last_became_peered":"2024-04-12T19:50:23.691571+0000","last_unstale":"2024-04-12T19:50:25.675732+0000","last_undegraded":"2024-04-12T19:50:25.675732+0000","last_fullsized":"2024-04-12T19:50:25.675732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:10:39.564249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193629+0000","last_change":"2024-04-12T19:50:24.690993+0000","last_active":"2024-04-12T19:50:26.193629+0000","last_peered":"2024-04-12T19:50:26.193629+0000","last_clean":"2024-04-12T19:50:26.193629+0000","last_became_active":"2024-04-12T19:50:24.690721+0000","last_became_peered":"2024-04-12T19:50:24.690721+0000","last_unstale":"2024-04-12T19:50:26.193629+0000","last_undegraded":"2024-04-12T19:50:26.193629+0000","last_fullsized":"2024-04-12T19:50:26.193629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184883+0000","last_change":"2024-04-12T19:50:23.689397+0000","last_active":"2024-04-12T19:50:26.184883+0000","last_peered":"2024-04-12T19:50:26.184883+0000","last_clean":"2024-04-12T19:50:26.184883+0000","last_became_active":"2024-04-12T19:50:23.688493+0000","last_became_peered":"2024-04-12T19:50:23.688493+0000","last_unstale":"2024-04-12T19:50:26.184883+0000","last_undegraded":"2024-04-12T19:50:26.184883+0000","last_fullsized":"2024-04-12T19:50:26.184883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682299+0000","last_change":"2024-04-12T19:50:24.667019+0000","last_active":"2024-04-12T19:50:25.682299+0000","last_peered":"2024-04-12T19:50:25.682299+0000","last_clean":"2024-04-12T19:50:25.682299+0000","last_became_active":"2024-04-12T19:50:24.666763+0000","last_became_peered":"2024-04-12T19:50:24.666763+0000","last_unstale":"2024-04-12T19:50:25.682299+0000","last_undegraded":"2024-04-12T19:50:25.682299+0000","last_fullsized":"2024-04-12T19:50:25.682299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673739+0000","last_change":"2024-04-12T19:50:23.686496+0000","last_active":"2024-04-12T19:50:25.673739+0000","last_peered":"2024-04-12T19:50:25.673739+0000","last_clean":"2024-04-12T19:50:25.673739+0000","last_became_active":"2024-04-12T19:50:23.684913+0000","last_became_peered":"2024-04-12T19:50:23.684913+0000","last_unstale":"2024-04-12T19:50:25.673739+0000","last_undegraded":"2024-04-12T19:50:25.673739+0000","last_fullsized":"2024-04-12T19:50:25.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676067+0000","last_change":"2024-04-12T19:50:24.693409+0000","last_active":"2024-04-12T19:50:25.676067+0000","last_peered":"2024-04-12T19:50:25.676067+0000","last_clean":"2024-04-12T19:50:25.676067+0000","last_became_active":"2024-04-12T19:50:24.686066+0000","last_became_peered":"2024-04-12T19:50:24.686066+0000","last_unstale":"2024-04-12T19:50:25.676067+0000","last_undegraded":"2024-04-12T19:50:25.676067+0000","last_fullsized":"2024-04-12T19:50:25.676067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689282+0000","last_change":"2024-04-12T19:50:23.685660+0000","last_active":"2024-04-12T19:50:25.689282+0000","last_peered":"2024-04-12T19:50:25.689282+0000","last_clean":"2024-04-12T19:50:25.689282+0000","last_became_active":"2024-04-12T19:50:23.685112+0000","last_became_peered":"2024-04-12T19:50:23.685112+0000","last_unstale":"2024-04-12T19:50:25.689282+0000","last_undegraded":"2024-04-12T19:50:25.689282+0000","last_fullsized":"2024-04-12T19:50:25.689282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194160+0000","last_change":"2024-04-12T19:50:24.687686+0000","last_active":"2024-04-12T19:50:26.194160+0000","last_peered":"2024-04-12T19:50:26.194160+0000","last_clean":"2024-04-12T19:50:26.194160+0000","last_became_active":"2024-04-12T19:50:24.686557+0000","last_became_peered":"2024-04-12T19:50:24.686557+0000","last_unstale":"2024-04-12T19:50:26.194160+0000","last_undegraded":"2024-04-12T19:50:26.194160+0000","last_fullsized":"2024-04-12T19:50:26.194160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.697524+0000","last_change":"2024-04-12T19:50:23.688709+0000","last_active":"2024-04-12T19:50:24.697524+0000","last_peered":"2024-04-12T19:50:24.697524+0000","last_clean":"2024-04-12T19:50:24.697524+0000","last_became_active":"2024-04-12T19:50:23.687448+0000","last_became_peered":"2024-04-12T19:50:23.687448+0000","last_unstale":"2024-04-12T19:50:24.697524+0000","last_undegraded":"2024-04-12T19:50:24.697524+0000","last_fullsized":"2024-04-12T19:50:24.697524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:41.154672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184609+0000","last_change":"2024-04-12T19:50:24.692484+0000","last_active":"2024-04-12T19:50:26.184609+0000","last_peered":"2024-04-12T19:50:26.184609+0000","last_clean":"2024-04-12T19:50:26.184609+0000","last_became_active":"2024-04-12T19:50:24.692157+0000","last_became_peered":"2024-04-12T19:50:24.692157+0000","last_unstale":"2024-04-12T19:50:26.184609+0000","last_undegraded":"2024-04-12T19:50:26.184609+0000","last_fullsized":"2024-04-12T19:50:26.184609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:51:48.663863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675846+0000","last_change":"2024-04-12T19:50:23.686058+0000","last_active":"2024-04-12T19:50:25.675846+0000","last_peered":"2024-04-12T19:50:25.675846+0000","last_clean":"2024-04-12T19:50:25.675846+0000","last_became_active":"2024-04-12T19:50:23.685765+0000","last_became_peered":"2024-04-12T19:50:23.685765+0000","last_unstale":"2024-04-12T19:50:25.675846+0000","last_undegraded":"2024-04-12T19:50:25.675846+0000","last_fullsized":"2024-04-12T19:50:25.675846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:04:17.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688457+0000","last_change":"2024-04-12T19:50:24.682664+0000","last_active":"2024-04-12T19:50:25.688457+0000","last_peered":"2024-04-12T19:50:25.688457+0000","last_clean":"2024-04-12T19:50:25.688457+0000","last_became_active":"2024-04-12T19:50:24.682385+0000","last_became_peered":"2024-04-12T19:50:24.682385+0000","last_unstale":"2024-04-12T19:50:25.688457+0000","last_undegraded":"2024-04-12T19:50:25.688457+0000","last_fullsized":"2024-04-12T19:50:25.688457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.701218+0000","last_change":"2024-04-12T19:50:23.689886+0000","last_active":"2024-04-12T19:50:24.701218+0000","last_peered":"2024-04-12T19:50:24.701218+0000","last_clean":"2024-04-12T19:50:24.701218+0000","last_became_active":"2024-04-12T19:50:23.689536+0000","last_became_peered":"2024-04-12T19:50:23.689536+0000","last_unstale":"2024-04-12T19:50:24.701218+0000","last_undegraded":"2024-04-12T19:50:24.701218+0000","last_fullsized":"2024-04-12T19:50:24.701218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:30.521133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675832+0000","last_change":"2024-04-12T19:50:24.689487+0000","last_active":"2024-04-12T19:50:25.675832+0000","last_peered":"2024-04-12T19:50:25.675832+0000","last_clean":"2024-04-12T19:50:25.675832+0000","last_became_active":"2024-04-12T19:50:24.687365+0000","last_became_peered":"2024-04-12T19:50:24.687365+0000","last_unstale":"2024-04-12T19:50:25.675832+0000","last_undegraded":"2024-04-12T19:50:25.675832+0000","last_fullsized":"2024-04-12T19:50:25.675832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184932+0000","last_change":"2024-04-12T19:50:23.682121+0000","last_active":"2024-04-12T19:50:26.184932+0000","last_peered":"2024-04-12T19:50:26.184932+0000","last_clean":"2024-04-12T19:50:26.184932+0000","last_became_active":"2024-04-12T19:50:23.681803+0000","last_became_peered":"2024-04-12T19:50:23.681803+0000","last_unstale":"2024-04-12T19:50:26.184932+0000","last_undegraded":"2024-04-12T19:50:26.184932+0000","last_fullsized":"2024-04-12T19:50:26.184932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184717+0000","last_change":"2024-04-12T19:50:24.685212+0000","last_active":"2024-04-12T19:50:26.184717+0000","last_peered":"2024-04-12T19:50:26.184717+0000","last_clean":"2024-04-12T19:50:26.184717+0000","last_became_active":"2024-04-12T19:50:24.684863+0000","last_became_peered":"2024-04-12T19:50:24.684863+0000","last_unstale":"2024-04-12T19:50:26.184717+0000","last_undegraded":"2024-04-12T19:50:26.184717+0000","last_fullsized":"2024-04-12T19:50:26.184717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:36:27.562288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687944+0000","last_change":"2024-04-12T19:50:23.673596+0000","last_active":"2024-04-12T19:50:25.687944+0000","last_peered":"2024-04-12T19:50:25.687944+0000","last_clean":"2024-04-12T19:50:25.687944+0000","last_became_active":"2024-04-12T19:50:23.672686+0000","last_became_peered":"2024-04-12T19:50:23.672686+0000","last_unstale":"2024-04-12T19:50:25.687944+0000","last_undegraded":"2024-04-12T19:50:25.687944+0000","last_fullsized":"2024-04-12T19:50:25.687944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672719+0000","last_change":"2024-04-12T19:50:24.693699+0000","last_active":"2024-04-12T19:50:25.672719+0000","last_peered":"2024-04-12T19:50:25.672719+0000","last_clean":"2024-04-12T19:50:25.672719+0000","last_became_active":"2024-04-12T19:50:24.693376+0000","last_became_peered":"2024-04-12T19:50:24.693376+0000","last_unstale":"2024-04-12T19:50:25.672719+0000","last_undegraded":"2024-04-12T19:50:25.672719+0000","last_fullsized":"2024-04-12T19:50:25.672719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682161+0000","last_change":"2024-04-12T19:50:23.682371+0000","last_active":"2024-04-12T19:50:25.682161+0000","last_peered":"2024-04-12T19:50:25.682161+0000","last_clean":"2024-04-12T19:50:25.682161+0000","last_became_active":"2024-04-12T19:50:23.681683+0000","last_became_peered":"2024-04-12T19:50:23.681683+0000","last_unstale":"2024-04-12T19:50:25.682161+0000","last_undegraded":"2024-04-12T19:50:25.682161+0000","last_fullsized":"2024-04-12T19:50:25.682161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184368+0000","last_change":"2024-04-12T19:50:24.673851+0000","last_active":"2024-04-12T19:50:26.184368+0000","last_peered":"2024-04-12T19:50:26.184368+0000","last_clean":"2024-04-12T19:50:26.184368+0000","last_became_active":"2024-04-12T19:50:24.673551+0000","last_became_peered":"2024-04-12T19:50:24.673551+0000","last_unstale":"2024-04-12T19:50:26.184368+0000","last_undegraded":"2024-04-12T19:50:26.184368+0000","last_fullsized":"2024-04-12T19:50:26.184368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:29:34.148711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689481+0000","last_change":"2024-04-12T19:50:23.691929+0000","last_active":"2024-04-12T19:50:25.689481+0000","last_peered":"2024-04-12T19:50:25.689481+0000","last_clean":"2024-04-12T19:50:25.689481+0000","last_became_active":"2024-04-12T19:50:23.684811+0000","last_became_peered":"2024-04-12T19:50:23.684811+0000","last_unstale":"2024-04-12T19:50:25.689481+0000","last_undegraded":"2024-04-12T19:50:25.689481+0000","last_fullsized":"2024-04-12T19:50:25.689481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676370+0000","last_change":"2024-04-12T19:50:24.691614+0000","last_active":"2024-04-12T19:50:25.676370+0000","last_peered":"2024-04-12T19:50:25.676370+0000","last_clean":"2024-04-12T19:50:25.676370+0000","last_became_active":"2024-04-12T19:50:24.686194+0000","last_became_peered":"2024-04-12T19:50:24.686194+0000","last_unstale":"2024-04-12T19:50:25.676370+0000","last_undegraded":"2024-04-12T19:50:25.676370+0000","last_fullsized":"2024-04-12T19:50:25.676370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683033+0000","last_change":"2024-04-12T19:50:23.685902+0000","last_active":"2024-04-12T19:50:25.683033+0000","last_peered":"2024-04-12T19:50:25.683033+0000","last_clean":"2024-04-12T19:50:25.683033+0000","last_became_active":"2024-04-12T19:50:23.685614+0000","last_became_peered":"2024-04-12T19:50:23.685614+0000","last_unstale":"2024-04-12T19:50:25.683033+0000","last_undegraded":"2024-04-12T19:50:25.683033+0000","last_fullsized":"2024-04-12T19:50:25.683033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689356+0000","last_change":"2024-04-12T19:50:24.674299+0000","last_active":"2024-04-12T19:50:25.689356+0000","last_peered":"2024-04-12T19:50:25.689356+0000","last_clean":"2024-04-12T19:50:25.689356+0000","last_became_active":"2024-04-12T19:50:24.672616+0000","last_became_peered":"2024-04-12T19:50:24.672616+0000","last_unstale":"2024-04-12T19:50:25.689356+0000","last_undegraded":"2024-04-12T19:50:25.689356+0000","last_fullsized":"2024-04-12T19:50:25.689356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194553+0000","last_change":"2024-04-12T19:50:23.685829+0000","last_active":"2024-04-12T19:50:26.194553+0000","last_peered":"2024-04-12T19:50:26.194553+0000","last_clean":"2024-04-12T19:50:26.194553+0000","last_became_active":"2024-04-12T19:50:23.684944+0000","last_became_peered":"2024-04-12T19:50:23.684944+0000","last_unstale":"2024-04-12T19:50:26.194553+0000","last_undegraded":"2024-04-12T19:50:26.194553+0000","last_fullsized":"2024-04-12T19:50:26.194553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688739+0000","last_change":"2024-04-12T19:50:24.680192+0000","last_active":"2024-04-12T19:50:25.688739+0000","last_peered":"2024-04-12T19:50:25.688739+0000","last_clean":"2024-04-12T19:50:25.688739+0000","last_became_active":"2024-04-12T19:50:24.679916+0000","last_became_peered":"2024-04-12T19:50:24.679916+0000","last_unstale":"2024-04-12T19:50:25.688739+0000","last_undegraded":"2024-04-12T19:50:25.688739+0000","last_fullsized":"2024-04-12T19:50:25.688739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676558+0000","last_change":"2024-04-12T19:50:23.693367+0000","last_active":"2024-04-12T19:50:25.676558+0000","last_peered":"2024-04-12T19:50:25.676558+0000","last_clean":"2024-04-12T19:50:25.676558+0000","last_became_active":"2024-04-12T19:50:23.693036+0000","last_became_peered":"2024-04-12T19:50:23.693036+0000","last_unstale":"2024-04-12T19:50:25.676558+0000","last_undegraded":"2024-04-12T19:50:25.676558+0000","last_fullsized":"2024-04-12T19:50:25.676558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185470+0000","last_change":"2024-04-12T19:50:24.674923+0000","last_active":"2024-04-12T19:50:26.185470+0000","last_peered":"2024-04-12T19:50:26.185470+0000","last_clean":"2024-04-12T19:50:26.185470+0000","last_became_active":"2024-04-12T19:50:24.674524+0000","last_became_peered":"2024-04-12T19:50:24.674524+0000","last_unstale":"2024-04-12T19:50:26.185470+0000","last_undegraded":"2024-04-12T19:50:26.185470+0000","last_fullsized":"2024-04-12T19:50:26.185470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:56.439740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672856+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:50:25.672856+0000","last_peered":"2024-04-12T19:50:25.672856+0000","last_clean":"2024-04-12T19:50:25.672856+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:50:25.672856+0000","last_undegraded":"2024-04-12T19:50:25.672856+0000","last_fullsized":"2024-04-12T19:50:25.672856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184860+0000","last_change":"2024-04-12T19:50:23.676923+0000","last_active":"2024-04-12T19:50:26.184860+0000","last_peered":"2024-04-12T19:50:26.184860+0000","last_clean":"2024-04-12T19:50:26.184860+0000","last_became_active":"2024-04-12T19:50:23.676667+0000","last_became_peered":"2024-04-12T19:50:23.676667+0000","last_unstale":"2024-04-12T19:50:26.184860+0000","last_undegraded":"2024-04-12T19:50:26.184860+0000","last_fullsized":"2024-04-12T19:50:26.184860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:29.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682938+0000","last_change":"2024-04-12T19:50:24.673694+0000","last_active":"2024-04-12T19:50:25.682938+0000","last_peered":"2024-04-12T19:50:25.682938+0000","last_clean":"2024-04-12T19:50:25.682938+0000","last_became_active":"2024-04-12T19:50:24.673340+0000","last_became_peered":"2024-04-12T19:50:24.673340+0000","last_unstale":"2024-04-12T19:50:25.682938+0000","last_undegraded":"2024-04-12T19:50:25.682938+0000","last_fullsized":"2024-04-12T19:50:25.682938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184624+0000","last_change":"2024-04-12T19:50:23.690110+0000","last_active":"2024-04-12T19:50:26.184624+0000","last_peered":"2024-04-12T19:50:26.184624+0000","last_clean":"2024-04-12T19:50:26.184624+0000","last_became_active":"2024-04-12T19:50:23.689677+0000","last_became_peered":"2024-04-12T19:50:23.689677+0000","last_unstale":"2024-04-12T19:50:26.184624+0000","last_undegraded":"2024-04-12T19:50:26.184624+0000","last_fullsized":"2024-04-12T19:50:26.184624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:02.364998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676043+0000","last_change":"2024-04-12T19:50:24.675131+0000","last_active":"2024-04-12T19:50:25.676043+0000","last_peered":"2024-04-12T19:50:25.676043+0000","last_clean":"2024-04-12T19:50:25.676043+0000","last_became_active":"2024-04-12T19:50:24.674486+0000","last_became_peered":"2024-04-12T19:50:24.674486+0000","last_unstale":"2024-04-12T19:50:25.676043+0000","last_undegraded":"2024-04-12T19:50:25.676043+0000","last_fullsized":"2024-04-12T19:50:25.676043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:40.879093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.663820+0000","last_change":"2024-04-12T19:50:23.690325+0000","last_active":"2024-04-12T19:50:24.663820+0000","last_peered":"2024-04-12T19:50:24.663820+0000","last_clean":"2024-04-12T19:50:24.663820+0000","last_became_active":"2024-04-12T19:50:23.688951+0000","last_became_peered":"2024-04-12T19:50:23.688951+0000","last_unstale":"2024-04-12T19:50:24.663820+0000","last_undegraded":"2024-04-12T19:50:24.663820+0000","last_fullsized":"2024-04-12T19:50:24.663820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:49:41.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193423+0000","last_change":"2024-04-12T19:50:24.698245+0000","last_active":"2024-04-12T19:50:26.193423+0000","last_peered":"2024-04-12T19:50:26.193423+0000","last_clean":"2024-04-12T19:50:26.193423+0000","last_became_active":"2024-04-12T19:50:24.697937+0000","last_became_peered":"2024-04-12T19:50:24.697937+0000","last_unstale":"2024-04-12T19:50:26.193423+0000","last_undegraded":"2024-04-12T19:50:26.193423+0000","last_fullsized":"2024-04-12T19:50:26.193423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185039+0000","last_change":"2024-04-12T19:50:23.689007+0000","last_active":"2024-04-12T19:50:26.185039+0000","last_peered":"2024-04-12T19:50:26.185039+0000","last_clean":"2024-04-12T19:50:26.185039+0000","last_became_active":"2024-04-12T19:50:23.688654+0000","last_became_peered":"2024-04-12T19:50:23.688654+0000","last_unstale":"2024-04-12T19:50:26.185039+0000","last_undegraded":"2024-04-12T19:50:26.185039+0000","last_fullsized":"2024-04-12T19:50:26.185039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:24.585940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688821+0000","last_change":"2024-04-12T19:50:24.680562+0000","last_active":"2024-04-12T19:50:25.688821+0000","last_peered":"2024-04-12T19:50:25.688821+0000","last_clean":"2024-04-12T19:50:25.688821+0000","last_became_active":"2024-04-12T19:50:24.680276+0000","last_became_peered":"2024-04-12T19:50:24.680276+0000","last_unstale":"2024-04-12T19:50:25.688821+0000","last_undegraded":"2024-04-12T19:50:25.688821+0000","last_fullsized":"2024-04-12T19:50:25.688821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193774+0000","last_change":"2024-04-12T19:50:23.685407+0000","last_active":"2024-04-12T19:50:26.193774+0000","last_peered":"2024-04-12T19:50:26.193774+0000","last_clean":"2024-04-12T19:50:26.193774+0000","last_became_active":"2024-04-12T19:50:23.684661+0000","last_became_peered":"2024-04-12T19:50:23.684661+0000","last_unstale":"2024-04-12T19:50:26.193774+0000","last_undegraded":"2024-04-12T19:50:26.193774+0000","last_fullsized":"2024-04-12T19:50:26.193774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675625+0000","last_change":"2024-04-12T19:50:24.686111+0000","last_active":"2024-04-12T19:50:25.675625+0000","last_peered":"2024-04-12T19:50:25.675625+0000","last_clean":"2024-04-12T19:50:25.675625+0000","last_became_active":"2024-04-12T19:50:24.685534+0000","last_became_peered":"2024-04-12T19:50:24.685534+0000","last_unstale":"2024-04-12T19:50:25.675625+0000","last_undegraded":"2024-04-12T19:50:25.675625+0000","last_fullsized":"2024-04-12T19:50:25.675625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185739+0000","last_change":"2024-04-12T19:50:23.676900+0000","last_active":"2024-04-12T19:50:26.185739+0000","last_peered":"2024-04-12T19:50:26.185739+0000","last_clean":"2024-04-12T19:50:26.185739+0000","last_became_active":"2024-04-12T19:50:23.676612+0000","last_became_peered":"2024-04-12T19:50:23.676612+0000","last_unstale":"2024-04-12T19:50:26.185739+0000","last_undegraded":"2024-04-12T19:50:26.185739+0000","last_fullsized":"2024-04-12T19:50:26.185739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:39.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676360+0000","last_change":"2024-04-12T19:50:24.695675+0000","last_active":"2024-04-12T19:50:25.676360+0000","last_peered":"2024-04-12T19:50:25.676360+0000","last_clean":"2024-04-12T19:50:25.676360+0000","last_became_active":"2024-04-12T19:50:24.695245+0000","last_became_peered":"2024-04-12T19:50:24.695245+0000","last_unstale":"2024-04-12T19:50:25.676360+0000","last_undegraded":"2024-04-12T19:50:25.676360+0000","last_fullsized":"2024-04-12T19:50:25.676360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.686136+0000","last_change":"2024-04-12T19:50:23.681572+0000","last_active":"2024-04-12T19:50:25.686136+0000","last_peered":"2024-04-12T19:50:25.686136+0000","last_clean":"2024-04-12T19:50:25.686136+0000","last_became_active":"2024-04-12T19:50:23.679612+0000","last_became_peered":"2024-04-12T19:50:23.679612+0000","last_unstale":"2024-04-12T19:50:25.686136+0000","last_undegraded":"2024-04-12T19:50:25.686136+0000","last_fullsized":"2024-04-12T19:50:25.686136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682950+0000","last_change":"2024-04-12T19:50:24.686596+0000","last_active":"2024-04-12T19:50:25.682950+0000","last_peered":"2024-04-12T19:50:25.682950+0000","last_clean":"2024-04-12T19:50:25.682950+0000","last_became_active":"2024-04-12T19:50:24.685023+0000","last_became_peered":"2024-04-12T19:50:24.685023+0000","last_unstale":"2024-04-12T19:50:25.682950+0000","last_undegraded":"2024-04-12T19:50:25.682950+0000","last_fullsized":"2024-04-12T19:50:25.682950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185887+0000","last_change":"2024-04-12T19:50:23.684680+0000","last_active":"2024-04-12T19:50:26.185887+0000","last_peered":"2024-04-12T19:50:26.185887+0000","last_clean":"2024-04-12T19:50:26.185887+0000","last_became_active":"2024-04-12T19:50:23.684298+0000","last_became_peered":"2024-04-12T19:50:23.684298+0000","last_unstale":"2024-04-12T19:50:26.185887+0000","last_undegraded":"2024-04-12T19:50:26.185887+0000","last_fullsized":"2024-04-12T19:50:26.185887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:06.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683078+0000","last_change":"2024-04-12T19:50:24.676462+0000","last_active":"2024-04-12T19:50:25.683078+0000","last_peered":"2024-04-12T19:50:25.683078+0000","last_clean":"2024-04-12T19:50:25.683078+0000","last_became_active":"2024-04-12T19:50:24.676182+0000","last_became_peered":"2024-04-12T19:50:24.676182+0000","last_unstale":"2024-04-12T19:50:25.683078+0000","last_undegraded":"2024-04-12T19:50:25.683078+0000","last_fullsized":"2024-04-12T19:50:25.683078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687692+0000","last_change":"2024-04-12T19:50:23.684797+0000","last_active":"2024-04-12T19:50:25.687692+0000","last_peered":"2024-04-12T19:50:25.687692+0000","last_clean":"2024-04-12T19:50:25.687692+0000","last_became_active":"2024-04-12T19:50:23.678108+0000","last_became_peered":"2024-04-12T19:50:23.678108+0000","last_unstale":"2024-04-12T19:50:25.687692+0000","last_undegraded":"2024-04-12T19:50:25.687692+0000","last_fullsized":"2024-04-12T19:50:25.687692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184277+0000","last_change":"2024-04-12T19:50:24.681150+0000","last_active":"2024-04-12T19:50:26.184277+0000","last_peered":"2024-04-12T19:50:26.184277+0000","last_clean":"2024-04-12T19:50:26.184277+0000","last_became_active":"2024-04-12T19:50:24.680857+0000","last_became_peered":"2024-04-12T19:50:24.680857+0000","last_unstale":"2024-04-12T19:50:26.184277+0000","last_undegraded":"2024-04-12T19:50:26.184277+0000","last_fullsized":"2024-04-12T19:50:26.184277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682575+0000","last_change":"2024-04-12T19:50:23.691210+0000","last_active":"2024-04-12T19:50:25.682575+0000","last_peered":"2024-04-12T19:50:25.682575+0000","last_clean":"2024-04-12T19:50:25.682575+0000","last_became_active":"2024-04-12T19:50:23.690943+0000","last_became_peered":"2024-04-12T19:50:23.690943+0000","last_unstale":"2024-04-12T19:50:25.682575+0000","last_undegraded":"2024-04-12T19:50:25.682575+0000","last_fullsized":"2024-04-12T19:50:25.682575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682186+0000","last_change":"2024-04-12T19:50:24.667678+0000","last_active":"2024-04-12T19:50:25.682186+0000","last_peered":"2024-04-12T19:50:25.682186+0000","last_clean":"2024-04-12T19:50:25.682186+0000","last_became_active":"2024-04-12T19:50:24.667407+0000","last_became_peered":"2024-04-12T19:50:24.667407+0000","last_unstale":"2024-04-12T19:50:25.682186+0000","last_undegraded":"2024-04-12T19:50:25.682186+0000","last_fullsized":"2024-04-12T19:50:25.682186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676272+0000","last_change":"2024-04-12T19:50:23.690317+0000","last_active":"2024-04-12T19:50:25.676272+0000","last_peered":"2024-04-12T19:50:25.676272+0000","last_clean":"2024-04-12T19:50:25.676272+0000","last_became_active":"2024-04-12T19:50:23.689037+0000","last_became_peered":"2024-04-12T19:50:23.689037+0000","last_unstale":"2024-04-12T19:50:25.676272+0000","last_undegraded":"2024-04-12T19:50:25.676272+0000","last_fullsized":"2024-04-12T19:50:25.676272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.695833+0000","last_change":"2024-04-12T19:50:24.681363+0000","last_active":"2024-04-12T19:50:24.695833+0000","last_peered":"2024-04-12T19:50:24.695833+0000","last_clean":"2024-04-12T19:50:24.695833+0000","last_became_active":"2024-04-12T19:50:24.681100+0000","last_became_peered":"2024-04-12T19:50:24.681100+0000","last_unstale":"2024-04-12T19:50:24.695833+0000","last_undegraded":"2024-04-12T19:50:24.695833+0000","last_fullsized":"2024-04-12T19:50:24.695833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:41:07.324095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184678+0000","last_change":"2024-04-12T19:50:23.679805+0000","last_active":"2024-04-12T19:50:26.184678+0000","last_peered":"2024-04-12T19:50:26.184678+0000","last_clean":"2024-04-12T19:50:26.184678+0000","last_became_active":"2024-04-12T19:50:23.679501+0000","last_became_peered":"2024-04-12T19:50:23.679501+0000","last_unstale":"2024-04-12T19:50:26.184678+0000","last_undegraded":"2024-04-12T19:50:26.184678+0000","last_fullsized":"2024-04-12T19:50:26.184678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:44.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184641+0000","last_change":"2024-04-12T19:50:24.678712+0000","last_active":"2024-04-12T19:50:26.184641+0000","last_peered":"2024-04-12T19:50:26.184641+0000","last_clean":"2024-04-12T19:50:26.184641+0000","last_became_active":"2024-04-12T19:50:24.677584+0000","last_became_peered":"2024-04-12T19:50:24.677584+0000","last_unstale":"2024-04-12T19:50:26.184641+0000","last_undegraded":"2024-04-12T19:50:26.184641+0000","last_fullsized":"2024-04-12T19:50:26.184641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:27:08.109541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185338+0000","last_change":"2024-04-12T19:50:24.698869+0000","last_active":"2024-04-12T19:50:26.185338+0000","last_peered":"2024-04-12T19:50:26.185338+0000","last_clean":"2024-04-12T19:50:26.185338+0000","last_became_active":"2024-04-12T19:50:24.698605+0000","last_became_peered":"2024-04-12T19:50:24.698605+0000","last_unstale":"2024-04-12T19:50:26.185338+0000","last_undegraded":"2024-04-12T19:50:26.185338+0000","last_fullsized":"2024-04-12T19:50:26.185338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:26:54.420503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688867+0000","last_change":"2024-04-12T19:50:23.678658+0000","last_active":"2024-04-12T19:50:25.688867+0000","last_peered":"2024-04-12T19:50:25.688867+0000","last_clean":"2024-04-12T19:50:25.688867+0000","last_became_active":"2024-04-12T19:50:23.676520+0000","last_became_peered":"2024-04-12T19:50:23.676520+0000","last_unstale":"2024-04-12T19:50:25.688867+0000","last_undegraded":"2024-04-12T19:50:25.688867+0000","last_fullsized":"2024-04-12T19:50:25.688867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673494+0000","last_change":"2024-04-12T19:50:24.684296+0000","last_active":"2024-04-12T19:50:25.673494+0000","last_peered":"2024-04-12T19:50:25.673494+0000","last_clean":"2024-04-12T19:50:25.673494+0000","last_became_active":"2024-04-12T19:50:24.683867+0000","last_became_peered":"2024-04-12T19:50:24.683867+0000","last_unstale":"2024-04-12T19:50:25.673494+0000","last_undegraded":"2024-04-12T19:50:25.673494+0000","last_fullsized":"2024-04-12T19:50:25.673494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688543+0000","last_change":"2024-04-12T19:50:23.680894+0000","last_active":"2024-04-12T19:50:25.688543+0000","last_peered":"2024-04-12T19:50:25.688543+0000","last_clean":"2024-04-12T19:50:25.688543+0000","last_became_active":"2024-04-12T19:50:23.679663+0000","last_became_peered":"2024-04-12T19:50:23.679663+0000","last_unstale":"2024-04-12T19:50:25.688543+0000","last_undegraded":"2024-04-12T19:50:25.688543+0000","last_fullsized":"2024-04-12T19:50:25.688543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194138+0000","last_change":"2024-04-12T19:50:24.695055+0000","last_active":"2024-04-12T19:50:26.194138+0000","last_peered":"2024-04-12T19:50:26.194138+0000","last_clean":"2024-04-12T19:50:26.194138+0000","last_became_active":"2024-04-12T19:50:24.694565+0000","last_became_peered":"2024-04-12T19:50:24.694565+0000","last_unstale":"2024-04-12T19:50:26.194138+0000","last_undegraded":"2024-04-12T19:50:26.194138+0000","last_fullsized":"2024-04-12T19:50:26.194138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194545+0000","last_change":"2024-04-12T19:50:23.684953+0000","last_active":"2024-04-12T19:50:26.194545+0000","last_peered":"2024-04-12T19:50:26.194545+0000","last_clean":"2024-04-12T19:50:26.194545+0000","last_became_active":"2024-04-12T19:50:23.684555+0000","last_became_peered":"2024-04-12T19:50:23.684555+0000","last_unstale":"2024-04-12T19:50:26.194545+0000","last_undegraded":"2024-04-12T19:50:26.194545+0000","last_fullsized":"2024-04-12T19:50:26.194545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194574+0000","last_change":"2024-04-12T19:50:24.701879+0000","last_active":"2024-04-12T19:50:26.194574+0000","last_peered":"2024-04-12T19:50:26.194574+0000","last_clean":"2024-04-12T19:50:26.194574+0000","last_became_active":"2024-04-12T19:50:24.701573+0000","last_became_peered":"2024-04-12T19:50:24.701573+0000","last_unstale":"2024-04-12T19:50:26.194574+0000","last_undegraded":"2024-04-12T19:50:26.194574+0000","last_fullsized":"2024-04-12T19:50:26.194574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683114+0000","last_change":"2024-04-12T19:50:23.673049+0000","last_active":"2024-04-12T19:50:25.683114+0000","last_peered":"2024-04-12T19:50:25.683114+0000","last_clean":"2024-04-12T19:50:25.683114+0000","last_became_active":"2024-04-12T19:50:23.672610+0000","last_became_peered":"2024-04-12T19:50:23.672610+0000","last_unstale":"2024-04-12T19:50:25.683114+0000","last_undegraded":"2024-04-12T19:50:25.683114+0000","last_fullsized":"2024-04-12T19:50:25.683114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":70566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:50:29.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:50:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.457+0000 7f588e200640 1 Processor -- start 2024-04-12T19:50:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.457+0000 7f588e200640 1 -- start start 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f588e200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58881461f0 0x7f58880ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f588e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f588e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58880b4be0 con 0x7f58880cedf0 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f588e200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f58880a9640 con 0x7f58881461f0 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f588e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58880cf250 con 0x7f58880b3fe0 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58881461f0 0x7f58880ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58881461f0 0x7f58880ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60920/0 (socket says 172.21.15.32:60920) 2024-04-12T19:50:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 -- 172.21.15.32:0/1866959128 learned_addr learned my addr 172.21.15.32:0/1866959128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 -- 172.21.15.32:0/1866959128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 msgr2=0x7f588815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 --2- 172.21.15.32:0/1866959128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 -- 172.21.15.32:0/1866959128 >> v1:172.21.15.169:6789/0 conn(0x7f58880cedf0 legacy=0x7f58880b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.459+0000 7f5887400640 1 -- 172.21.15.32:0/1866959128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58880b4620 con 0x7f58881461f0 2024-04-12T19:50:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.461+0000 7f5887400640 1 --2- 172.21.15.32:0/1866959128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58881461f0 0x7f58880ce8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5878009870 tx=0x7f587802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af13a52d293686c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.461+0000 7f5886a00640 1 -- 172.21.15.32:0/1866959128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878034070 con 0x7f58881461f0 2024-04-12T19:50:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.461+0000 7f5886a00640 1 -- 172.21.15.32:0/1866959128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878030840 con 0x7f58881461f0 2024-04-12T19:50:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.461+0000 7f5886a00640 1 -- 172.21.15.32:0/1866959128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878030c10 con 0x7f58881461f0 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 -- 172.21.15.32:0/1866959128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58881461f0 msgr2=0x7f58880ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 --2- 172.21.15.32:0/1866959128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58881461f0 0x7f58880ce8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5878009870 tx=0x7f587802b980 comp rx=0 tx=0).stop 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 -- 172.21.15.32:0/1866959128 shutdown_connections 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 --2- 172.21.15.32:0/1866959128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 --2- 172.21.15.32:0/1866959128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58881461f0 0x7f58880ce8b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 -- 172.21.15.32:0/1866959128 >> 172.21.15.32:0/1866959128 conn(0x7f58880b27b0 msgr2=0x7f58880b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 -- 172.21.15.32:0/1866959128 shutdown_connections 2024-04-12T19:50:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.462+0000 7f588e200640 1 -- 172.21.15.32:0/1866959128 wait complete. 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.464+0000 7f588e200640 1 Processor -- start 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.464+0000 7f588e200640 1 -- start start 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588813f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58881461f0 0x7f5888143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f588815d290 con 0x7f58880cedf0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f588815cfb0 con 0x7f58881461f0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f588e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58881465d0 con 0x7f58880b3fe0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f587fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f587fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60936/0 (socket says 172.21.15.32:60936) 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f587fe00640 1 -- 172.21.15.32:0/1892674100 learned_addr learned my addr 172.21.15.32:0/1892674100 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f5887400640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588813f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.465+0000 7f5887e00640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58881461f0 0x7f5888143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 -- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 msgr2=0x7f588813f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588813f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 -- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58881461f0 msgr2=0x7f5888143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58881461f0 0x7f5888143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 -- 172.21.15.32:0/1892674100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881e7f30 con 0x7f58880cedf0 2024-04-12T19:50:29.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f5887400640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587fe00640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f587000c430 tx=0x7f587000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870011790 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870017070 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870011bb0 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881e8210 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.466+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881e87d0 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.468+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5888068c80 con 0x7f58880cedf0 2024-04-12T19:50:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.468+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f587001c460 con 0x7f58880cedf0 2024-04-12T19:50:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.469+0000 7f587f400640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 0x7f586406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.469+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f587008b990 con 0x7f58880cedf0 2024-04-12T19:50:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.470+0000 7f5887400640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 0x7f586406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.471+0000 7f5887400640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 0x7f586406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5878036000 tx=0x7f587803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.474+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5870059e50 con 0x7f58880cedf0 2024-04-12T19:50:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.680+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f588815ca40 con 0x7f586406c8f0 2024-04-12T19:50:29.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.698+0000 7f587f400640 1 -- 172.21.15.32:0/1892674100 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f588815ca40 con 0x7f586406c8f0 2024-04-12T19:50:29.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:50:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.704+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 msgr2=0x7f586406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.704+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 0x7f586406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5878036000 tx=0x7f587803b040 comp rx=0 tx=0).stop 2024-04-12T19:50:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.704+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 msgr2=0x7f58881e95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.704+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f587000c430 tx=0x7f587000c8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 shutdown_connections 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f586406c8f0 0x7f586406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58881461f0 0x7f5888143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58880cedf0 0x7f58881e95e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 --2- 172.21.15.32:0/1892674100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58880b3fe0 0x7f588813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 >> 172.21.15.32:0/1892674100 conn(0x7f58880b27b0 msgr2=0x7f58880c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 shutdown_connections 2024-04-12T19:50:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.705+0000 7f588e200640 1 -- 172.21.15.32:0/1892674100 wait complete. 2024-04-12T19:50:29.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-12T19:50:29.626669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235744,"kb_used_data":2928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738976,"statfs":{"total":773094113280,"available":772852711424,"internally_reserved":0,"allocated":2998272,"data_stored":1783228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001917"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689002+0000","last_change":"2024-04-12T19:50:24.687884+0000","last_active":"2024-04-12T19:50:25.689002+0000","last_peered":"2024-04-12T19:50:25.689002+0000","last_clean":"2024-04-12T19:50:25.689002+0000","last_became_active":"2024-04-12T19:50:24.686545+0000","last_became_peered":"2024-04-12T19:50:24.686545+0000","last_unstale":"2024-04-12T19:50:25.689002+0000","last_undegraded":"2024-04-12T19:50:25.689002+0000","last_fullsized":"2024-04-12T19:50:25.689002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194273+0000","last_change":"2024-04-12T19:50:24.693720+0000","last_active":"2024-04-12T19:50:26.194273+0000","last_peered":"2024-04-12T19:50:26.194273+0000","last_clean":"2024-04-12T19:50:26.194273+0000","last_became_active":"2024-04-12T19:50:24.693332+0000","last_became_peered":"2024-04-12T19:50:24.693332+0000","last_unstale":"2024-04-12T19:50:26.194273+0000","last_undegraded":"2024-04-12T19:50:26.194273+0000","last_fullsized":"2024-04-12T19:50:26.194273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194432+0000","last_change":"2024-04-12T19:50:24.688591+0000","last_active":"2024-04-12T19:50:26.194432+0000","last_peered":"2024-04-12T19:50:26.194432+0000","last_clean":"2024-04-12T19:50:26.194432+0000","last_became_active":"2024-04-12T19:50:24.688345+0000","last_became_peered":"2024-04-12T19:50:24.688345+0000","last_unstale":"2024-04-12T19:50:26.194432+0000","last_undegraded":"2024-04-12T19:50:26.194432+0000","last_fullsized":"2024-04-12T19:50:26.194432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675845+0000","last_change":"2024-04-12T19:50:24.680503+0000","last_active":"2024-04-12T19:50:25.675845+0000","last_peered":"2024-04-12T19:50:25.675845+0000","last_clean":"2024-04-12T19:50:25.675845+0000","last_became_active":"2024-04-12T19:50:24.674820+0000","last_became_peered":"2024-04-12T19:50:24.674820+0000","last_unstale":"2024-04-12T19:50:25.675845+0000","last_undegraded":"2024-04-12T19:50:25.675845+0000","last_fullsized":"2024-04-12T19:50:25.675845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688640+0000","last_change":"2024-04-12T19:50:24.688597+0000","last_active":"2024-04-12T19:50:25.688640+0000","last_peered":"2024-04-12T19:50:25.688640+0000","last_clean":"2024-04-12T19:50:25.688640+0000","last_became_active":"2024-04-12T19:50:24.688308+0000","last_became_peered":"2024-04-12T19:50:24.688308+0000","last_unstale":"2024-04-12T19:50:25.688640+0000","last_undegraded":"2024-04-12T19:50:25.688640+0000","last_fullsized":"2024-04-12T19:50:25.688640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675161+0000","last_change":"2024-04-12T19:50:24.680554+0000","last_active":"2024-04-12T19:50:25.675161+0000","last_peered":"2024-04-12T19:50:25.675161+0000","last_clean":"2024-04-12T19:50:25.675161+0000","last_became_active":"2024-04-12T19:50:24.675178+0000","last_became_peered":"2024-04-12T19:50:24.675178+0000","last_unstale":"2024-04-12T19:50:25.675161+0000","last_undegraded":"2024-04-12T19:50:25.675161+0000","last_fullsized":"2024-04-12T19:50:25.675161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688158+0000","last_change":"2024-04-12T19:50:24.677924+0000","last_active":"2024-04-12T19:50:25.688158+0000","last_peered":"2024-04-12T19:50:25.688158+0000","last_clean":"2024-04-12T19:50:25.688158+0000","last_became_active":"2024-04-12T19:50:24.677414+0000","last_became_peered":"2024-04-12T19:50:24.677414+0000","last_unstale":"2024-04-12T19:50:25.688158+0000","last_undegraded":"2024-04-12T19:50:25.688158+0000","last_fullsized":"2024-04-12T19:50:25.688158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193170+0000","last_change":"2024-04-12T19:50:24.690479+0000","last_active":"2024-04-12T19:50:26.193170+0000","last_peered":"2024-04-12T19:50:26.193170+0000","last_clean":"2024-04-12T19:50:26.193170+0000","last_became_active":"2024-04-12T19:50:24.690010+0000","last_became_peered":"2024-04-12T19:50:24.690010+0000","last_unstale":"2024-04-12T19:50:26.193170+0000","last_undegraded":"2024-04-12T19:50:26.193170+0000","last_fullsized":"2024-04-12T19:50:26.193170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673588+0000","last_change":"2024-04-12T19:50:24.679542+0000","last_active":"2024-04-12T19:50:25.673588+0000","last_peered":"2024-04-12T19:50:25.673588+0000","last_clean":"2024-04-12T19:50:25.673588+0000","last_became_active":"2024-04-12T19:50:24.678975+0000","last_became_peered":"2024-04-12T19:50:24.678975+0000","last_unstale":"2024-04-12T19:50:25.673588+0000","last_undegraded":"2024-04-12T19:50:25.673588+0000","last_fullsized":"2024-04-12T19:50:25.673588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688977+0000","last_change":"2024-04-12T19:50:24.681572+0000","last_active":"2024-04-12T19:50:25.688977+0000","last_peered":"2024-04-12T19:50:25.688977+0000","last_clean":"2024-04-12T19:50:25.688977+0000","last_became_active":"2024-04-12T19:50:24.681309+0000","last_became_peered":"2024-04-12T19:50:24.681309+0000","last_unstale":"2024-04-12T19:50:25.688977+0000","last_undegraded":"2024-04-12T19:50:25.688977+0000","last_fullsized":"2024-04-12T19:50:25.688977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688257+0000","last_change":"2024-04-12T19:50:24.678439+0000","last_active":"2024-04-12T19:50:25.688257+0000","last_peered":"2024-04-12T19:50:25.688257+0000","last_clean":"2024-04-12T19:50:25.688257+0000","last_became_active":"2024-04-12T19:50:24.677934+0000","last_became_peered":"2024-04-12T19:50:24.677934+0000","last_unstale":"2024-04-12T19:50:25.688257+0000","last_undegraded":"2024-04-12T19:50:25.688257+0000","last_fullsized":"2024-04-12T19:50:25.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688579+0000","last_change":"2024-04-12T19:50:24.683054+0000","last_active":"2024-04-12T19:50:25.688579+0000","last_peered":"2024-04-12T19:50:25.688579+0000","last_clean":"2024-04-12T19:50:25.688579+0000","last_became_active":"2024-04-12T19:50:24.682749+0000","last_became_peered":"2024-04-12T19:50:24.682749+0000","last_unstale":"2024-04-12T19:50:25.688579+0000","last_undegraded":"2024-04-12T19:50:25.688579+0000","last_fullsized":"2024-04-12T19:50:25.688579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675969+0000","last_change":"2024-04-12T19:50:24.700812+0000","last_active":"2024-04-12T19:50:25.675969+0000","last_peered":"2024-04-12T19:50:25.675969+0000","last_clean":"2024-04-12T19:50:25.675969+0000","last_became_active":"2024-04-12T19:50:24.700470+0000","last_became_peered":"2024-04-12T19:50:24.700470+0000","last_unstale":"2024-04-12T19:50:25.675969+0000","last_undegraded":"2024-04-12T19:50:25.675969+0000","last_fullsized":"2024-04-12T19:50:25.675969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676868+0000","last_change":"2024-04-12T19:50:24.678150+0000","last_active":"2024-04-12T19:50:25.676868+0000","last_peered":"2024-04-12T19:50:25.676868+0000","last_clean":"2024-04-12T19:50:25.676868+0000","last_became_active":"2024-04-12T19:50:24.677811+0000","last_became_peered":"2024-04-12T19:50:24.677811+0000","last_unstale":"2024-04-12T19:50:25.676868+0000","last_undegraded":"2024-04-12T19:50:25.676868+0000","last_fullsized":"2024-04-12T19:50:25.676868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193225+0000","last_change":"2024-04-12T19:50:24.701334+0000","last_active":"2024-04-12T19:50:26.193225+0000","last_peered":"2024-04-12T19:50:26.193225+0000","last_clean":"2024-04-12T19:50:26.193225+0000","last_became_active":"2024-04-12T19:50:24.701046+0000","last_became_peered":"2024-04-12T19:50:24.701046+0000","last_unstale":"2024-04-12T19:50:26.193225+0000","last_undegraded":"2024-04-12T19:50:26.193225+0000","last_fullsized":"2024-04-12T19:50:26.193225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184164+0000","last_change":"2024-04-12T19:50:24.678787+0000","last_active":"2024-04-12T19:50:26.184164+0000","last_peered":"2024-04-12T19:50:26.184164+0000","last_clean":"2024-04-12T19:50:26.184164+0000","last_became_active":"2024-04-12T19:50:24.678537+0000","last_became_peered":"2024-04-12T19:50:24.678537+0000","last_unstale":"2024-04-12T19:50:26.184164+0000","last_undegraded":"2024-04-12T19:50:26.184164+0000","last_fullsized":"2024-04-12T19:50:26.184164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185746+0000","last_change":"2024-04-12T19:50:24.680249+0000","last_active":"2024-04-12T19:50:26.185746+0000","last_peered":"2024-04-12T19:50:26.185746+0000","last_clean":"2024-04-12T19:50:26.185746+0000","last_became_active":"2024-04-12T19:50:24.679835+0000","last_became_peered":"2024-04-12T19:50:24.679835+0000","last_unstale":"2024-04-12T19:50:26.185746+0000","last_undegraded":"2024-04-12T19:50:26.185746+0000","last_fullsized":"2024-04-12T19:50:26.185746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:17:07.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.699165+0000","last_change":"2024-04-12T19:50:24.697806+0000","last_active":"2024-04-12T19:50:24.699165+0000","last_peered":"2024-04-12T19:50:24.699165+0000","last_clean":"2024-04-12T19:50:24.699165+0000","last_became_active":"2024-04-12T19:50:24.697515+0000","last_became_peered":"2024-04-12T19:50:24.697515+0000","last_unstale":"2024-04-12T19:50:24.699165+0000","last_undegraded":"2024-04-12T19:50:24.699165+0000","last_fullsized":"2024-04-12T19:50:24.699165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:06:20.109649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185806+0000","last_change":"2024-04-12T19:50:24.684475+0000","last_active":"2024-04-12T19:50:26.185806+0000","last_peered":"2024-04-12T19:50:26.185806+0000","last_clean":"2024-04-12T19:50:26.185806+0000","last_became_active":"2024-04-12T19:50:24.684052+0000","last_became_peered":"2024-04-12T19:50:24.684052+0000","last_unstale":"2024-04-12T19:50:26.185806+0000","last_undegraded":"2024-04-12T19:50:26.185806+0000","last_fullsized":"2024-04-12T19:50:26.185806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:19:38.175136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688476+0000","last_change":"2024-04-12T19:50:24.679515+0000","last_active":"2024-04-12T19:50:25.688476+0000","last_peered":"2024-04-12T19:50:25.688476+0000","last_clean":"2024-04-12T19:50:25.688476+0000","last_became_active":"2024-04-12T19:50:24.678757+0000","last_became_peered":"2024-04-12T19:50:24.678757+0000","last_unstale":"2024-04-12T19:50:25.688476+0000","last_undegraded":"2024-04-12T19:50:25.688476+0000","last_fullsized":"2024-04-12T19:50:25.688476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682423+0000","last_change":"2024-04-12T19:50:24.674490+0000","last_active":"2024-04-12T19:50:25.682423+0000","last_peered":"2024-04-12T19:50:25.682423+0000","last_clean":"2024-04-12T19:50:25.682423+0000","last_became_active":"2024-04-12T19:50:24.674131+0000","last_became_peered":"2024-04-12T19:50:24.674131+0000","last_unstale":"2024-04-12T19:50:25.682423+0000","last_undegraded":"2024-04-12T19:50:25.682423+0000","last_fullsized":"2024-04-12T19:50:25.682423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673126+0000","last_change":"2024-04-12T19:50:24.673011+0000","last_active":"2024-04-12T19:50:25.673126+0000","last_peered":"2024-04-12T19:50:25.673126+0000","last_clean":"2024-04-12T19:50:25.673126+0000","last_became_active":"2024-04-12T19:50:24.672490+0000","last_became_peered":"2024-04-12T19:50:24.672490+0000","last_unstale":"2024-04-12T19:50:25.673126+0000","last_undegraded":"2024-04-12T19:50:25.673126+0000","last_fullsized":"2024-04-12T19:50:25.673126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193541+0000","last_change":"2024-04-12T19:50:24.692657+0000","last_active":"2024-04-12T19:50:26.193541+0000","last_peered":"2024-04-12T19:50:26.193541+0000","last_clean":"2024-04-12T19:50:26.193541+0000","last_became_active":"2024-04-12T19:50:24.692335+0000","last_became_peered":"2024-04-12T19:50:24.692335+0000","last_unstale":"2024-04-12T19:50:26.193541+0000","last_undegraded":"2024-04-12T19:50:26.193541+0000","last_fullsized":"2024-04-12T19:50:26.193541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184179+0000","last_change":"2024-04-12T19:50:24.692524+0000","last_active":"2024-04-12T19:50:26.184179+0000","last_peered":"2024-04-12T19:50:26.184179+0000","last_clean":"2024-04-12T19:50:26.184179+0000","last_became_active":"2024-04-12T19:50:24.692239+0000","last_became_peered":"2024-04-12T19:50:24.692239+0000","last_unstale":"2024-04-12T19:50:26.184179+0000","last_undegraded":"2024-04-12T19:50:26.184179+0000","last_fullsized":"2024-04-12T19:50:26.184179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193671+0000","last_change":"2024-04-12T19:50:24.686815+0000","last_active":"2024-04-12T19:50:26.193671+0000","last_peered":"2024-04-12T19:50:26.193671+0000","last_clean":"2024-04-12T19:50:26.193671+0000","last_became_active":"2024-04-12T19:50:24.686192+0000","last_became_peered":"2024-04-12T19:50:24.686192+0000","last_unstale":"2024-04-12T19:50:26.193671+0000","last_undegraded":"2024-04-12T19:50:26.193671+0000","last_fullsized":"2024-04-12T19:50:26.193671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676094+0000","last_change":"2024-04-12T19:50:24.686489+0000","last_active":"2024-04-12T19:50:25.676094+0000","last_peered":"2024-04-12T19:50:25.676094+0000","last_clean":"2024-04-12T19:50:25.676094+0000","last_became_active":"2024-04-12T19:50:24.685753+0000","last_became_peered":"2024-04-12T19:50:24.685753+0000","last_unstale":"2024-04-12T19:50:25.676094+0000","last_undegraded":"2024-04-12T19:50:25.676094+0000","last_fullsized":"2024-04-12T19:50:25.676094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676600+0000","last_change":"2024-04-12T19:50:24.675093+0000","last_active":"2024-04-12T19:50:25.676600+0000","last_peered":"2024-04-12T19:50:25.676600+0000","last_clean":"2024-04-12T19:50:25.676600+0000","last_became_active":"2024-04-12T19:50:24.674798+0000","last_became_peered":"2024-04-12T19:50:24.674798+0000","last_unstale":"2024-04-12T19:50:25.676600+0000","last_undegraded":"2024-04-12T19:50:25.676600+0000","last_fullsized":"2024-04-12T19:50:25.676600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676605+0000","last_change":"2024-04-12T19:50:24.681961+0000","last_active":"2024-04-12T19:50:25.676605+0000","last_peered":"2024-04-12T19:50:25.676605+0000","last_clean":"2024-04-12T19:50:25.676605+0000","last_became_active":"2024-04-12T19:50:24.681596+0000","last_became_peered":"2024-04-12T19:50:24.681596+0000","last_unstale":"2024-04-12T19:50:25.676605+0000","last_undegraded":"2024-04-12T19:50:25.676605+0000","last_fullsized":"2024-04-12T19:50:25.676605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194168+0000","last_change":"2024-04-12T19:50:24.688114+0000","last_active":"2024-04-12T19:50:26.194168+0000","last_peered":"2024-04-12T19:50:26.194168+0000","last_clean":"2024-04-12T19:50:26.194168+0000","last_became_active":"2024-04-12T19:50:24.687826+0000","last_became_peered":"2024-04-12T19:50:24.687826+0000","last_unstale":"2024-04-12T19:50:26.194168+0000","last_undegraded":"2024-04-12T19:50:26.194168+0000","last_fullsized":"2024-04-12T19:50:26.194168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184379+0000","last_change":"2024-04-12T19:50:24.697023+0000","last_active":"2024-04-12T19:50:26.184379+0000","last_peered":"2024-04-12T19:50:26.184379+0000","last_clean":"2024-04-12T19:50:26.184379+0000","last_became_active":"2024-04-12T19:50:24.696739+0000","last_became_peered":"2024-04-12T19:50:24.696739+0000","last_unstale":"2024-04-12T19:50:26.184379+0000","last_undegraded":"2024-04-12T19:50:26.184379+0000","last_fullsized":"2024-04-12T19:50:26.184379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:46:45.869946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.680237+0000","last_change":"2024-04-12T19:50:24.675081+0000","last_active":"2024-04-12T19:50:24.680237+0000","last_peered":"2024-04-12T19:50:24.680237+0000","last_clean":"2024-04-12T19:50:24.680237+0000","last_became_active":"2024-04-12T19:50:24.674812+0000","last_became_peered":"2024-04-12T19:50:24.674812+0000","last_unstale":"2024-04-12T19:50:24.680237+0000","last_undegraded":"2024-04-12T19:50:24.680237+0000","last_fullsized":"2024-04-12T19:50:24.680237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:02:45.126635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.674913+0000","last_change":"2024-04-12T19:50:24.687854+0000","last_active":"2024-04-12T19:50:25.674913+0000","last_peered":"2024-04-12T19:50:25.674913+0000","last_clean":"2024-04-12T19:50:25.674913+0000","last_became_active":"2024-04-12T19:50:24.685605+0000","last_became_peered":"2024-04-12T19:50:24.685605+0000","last_unstale":"2024-04-12T19:50:25.674913+0000","last_undegraded":"2024-04-12T19:50:25.674913+0000","last_fullsized":"2024-04-12T19:50:25.674913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:11.962955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673270+0000","last_change":"2024-04-12T19:50:24.694100+0000","last_active":"2024-04-12T19:50:25.673270+0000","last_peered":"2024-04-12T19:50:25.673270+0000","last_clean":"2024-04-12T19:50:25.673270+0000","last_became_active":"2024-04-12T19:50:24.693828+0000","last_became_peered":"2024-04-12T19:50:24.693828+0000","last_unstale":"2024-04-12T19:50:25.673270+0000","last_undegraded":"2024-04-12T19:50:25.673270+0000","last_fullsized":"2024-04-12T19:50:25.673270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194122+0000","last_change":"2024-04-12T19:50:23.685737+0000","last_active":"2024-04-12T19:50:26.194122+0000","last_peered":"2024-04-12T19:50:26.194122+0000","last_clean":"2024-04-12T19:50:26.194122+0000","last_became_active":"2024-04-12T19:50:23.685371+0000","last_became_peered":"2024-04-12T19:50:23.685371+0000","last_unstale":"2024-04-12T19:50:26.194122+0000","last_undegraded":"2024-04-12T19:50:26.194122+0000","last_fullsized":"2024-04-12T19:50:26.194122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682544+0000","last_change":"2024-04-12T19:50:24.681008+0000","last_active":"2024-04-12T19:50:25.682544+0000","last_peered":"2024-04-12T19:50:25.682544+0000","last_clean":"2024-04-12T19:50:25.682544+0000","last_became_active":"2024-04-12T19:50:24.680689+0000","last_became_peered":"2024-04-12T19:50:24.680689+0000","last_unstale":"2024-04-12T19:50:25.682544+0000","last_undegraded":"2024-04-12T19:50:25.682544+0000","last_fullsized":"2024-04-12T19:50:25.682544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184687+0000","last_change":"2024-04-12T19:50:23.691933+0000","last_active":"2024-04-12T19:50:26.184687+0000","last_peered":"2024-04-12T19:50:26.184687+0000","last_clean":"2024-04-12T19:50:26.184687+0000","last_became_active":"2024-04-12T19:50:23.691622+0000","last_became_peered":"2024-04-12T19:50:23.691622+0000","last_unstale":"2024-04-12T19:50:26.184687+0000","last_undegraded":"2024-04-12T19:50:26.184687+0000","last_fullsized":"2024-04-12T19:50:26.184687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675732+0000","last_change":"2024-04-12T19:50:23.691962+0000","last_active":"2024-04-12T19:50:25.675732+0000","last_peered":"2024-04-12T19:50:25.675732+0000","last_clean":"2024-04-12T19:50:25.675732+0000","last_became_active":"2024-04-12T19:50:23.691571+0000","last_became_peered":"2024-04-12T19:50:23.691571+0000","last_unstale":"2024-04-12T19:50:25.675732+0000","last_undegraded":"2024-04-12T19:50:25.675732+0000","last_fullsized":"2024-04-12T19:50:25.675732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:10:39.564249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193629+0000","last_change":"2024-04-12T19:50:24.690993+0000","last_active":"2024-04-12T19:50:26.193629+0000","last_peered":"2024-04-12T19:50:26.193629+0000","last_clean":"2024-04-12T19:50:26.193629+0000","last_became_active":"2024-04-12T19:50:24.690721+0000","last_became_peered":"2024-04-12T19:50:24.690721+0000","last_unstale":"2024-04-12T19:50:26.193629+0000","last_undegraded":"2024-04-12T19:50:26.193629+0000","last_fullsized":"2024-04-12T19:50:26.193629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184883+0000","last_change":"2024-04-12T19:50:23.689397+0000","last_active":"2024-04-12T19:50:26.184883+0000","last_peered":"2024-04-12T19:50:26.184883+0000","last_clean":"2024-04-12T19:50:26.184883+0000","last_became_active":"2024-04-12T19:50:23.688493+0000","last_became_peered":"2024-04-12T19:50:23.688493+0000","last_unstale":"2024-04-12T19:50:26.184883+0000","last_undegraded":"2024-04-12T19:50:26.184883+0000","last_fullsized":"2024-04-12T19:50:26.184883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682299+0000","last_change":"2024-04-12T19:50:24.667019+0000","last_active":"2024-04-12T19:50:25.682299+0000","last_peered":"2024-04-12T19:50:25.682299+0000","last_clean":"2024-04-12T19:50:25.682299+0000","last_became_active":"2024-04-12T19:50:24.666763+0000","last_became_peered":"2024-04-12T19:50:24.666763+0000","last_unstale":"2024-04-12T19:50:25.682299+0000","last_undegraded":"2024-04-12T19:50:25.682299+0000","last_fullsized":"2024-04-12T19:50:25.682299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673739+0000","last_change":"2024-04-12T19:50:23.686496+0000","last_active":"2024-04-12T19:50:25.673739+0000","last_peered":"2024-04-12T19:50:25.673739+0000","last_clean":"2024-04-12T19:50:25.673739+0000","last_became_active":"2024-04-12T19:50:23.684913+0000","last_became_peered":"2024-04-12T19:50:23.684913+0000","last_unstale":"2024-04-12T19:50:25.673739+0000","last_undegraded":"2024-04-12T19:50:25.673739+0000","last_fullsized":"2024-04-12T19:50:25.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676067+0000","last_change":"2024-04-12T19:50:24.693409+0000","last_active":"2024-04-12T19:50:25.676067+0000","last_peered":"2024-04-12T19:50:25.676067+0000","last_clean":"2024-04-12T19:50:25.676067+0000","last_became_active":"2024-04-12T19:50:24.686066+0000","last_became_peered":"2024-04-12T19:50:24.686066+0000","last_unstale":"2024-04-12T19:50:25.676067+0000","last_undegraded":"2024-04-12T19:50:25.676067+0000","last_fullsized":"2024-04-12T19:50:25.676067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689282+0000","last_change":"2024-04-12T19:50:23.685660+0000","last_active":"2024-04-12T19:50:25.689282+0000","last_peered":"2024-04-12T19:50:25.689282+0000","last_clean":"2024-04-12T19:50:25.689282+0000","last_became_active":"2024-04-12T19:50:23.685112+0000","last_became_peered":"2024-04-12T19:50:23.685112+0000","last_unstale":"2024-04-12T19:50:25.689282+0000","last_undegraded":"2024-04-12T19:50:25.689282+0000","last_fullsized":"2024-04-12T19:50:25.689282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194160+0000","last_change":"2024-04-12T19:50:24.687686+0000","last_active":"2024-04-12T19:50:26.194160+0000","last_peered":"2024-04-12T19:50:26.194160+0000","last_clean":"2024-04-12T19:50:26.194160+0000","last_became_active":"2024-04-12T19:50:24.686557+0000","last_became_peered":"2024-04-12T19:50:24.686557+0000","last_unstale":"2024-04-12T19:50:26.194160+0000","last_undegraded":"2024-04-12T19:50:26.194160+0000","last_fullsized":"2024-04-12T19:50:26.194160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.697524+0000","last_change":"2024-04-12T19:50:23.688709+0000","last_active":"2024-04-12T19:50:24.697524+0000","last_peered":"2024-04-12T19:50:24.697524+0000","last_clean":"2024-04-12T19:50:24.697524+0000","last_became_active":"2024-04-12T19:50:23.687448+0000","last_became_peered":"2024-04-12T19:50:23.687448+0000","last_unstale":"2024-04-12T19:50:24.697524+0000","last_undegraded":"2024-04-12T19:50:24.697524+0000","last_fullsized":"2024-04-12T19:50:24.697524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:41.154672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184609+0000","last_change":"2024-04-12T19:50:24.692484+0000","last_active":"2024-04-12T19:50:26.184609+0000","last_peered":"2024-04-12T19:50:26.184609+0000","last_clean":"2024-04-12T19:50:26.184609+0000","last_became_active":"2024-04-12T19:50:24.692157+0000","last_became_peered":"2024-04-12T19:50:24.692157+0000","last_unstale":"2024-04-12T19:50:26.184609+0000","last_undegraded":"2024-04-12T19:50:26.184609+0000","last_fullsized":"2024-04-12T19:50:26.184609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:51:48.663863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675846+0000","last_change":"2024-04-12T19:50:23.686058+0000","last_active":"2024-04-12T19:50:25.675846+0000","last_peered":"2024-04-12T19:50:25.675846+0000","last_clean":"2024-04-12T19:50:25.675846+0000","last_became_active":"2024-04-12T19:50:23.685765+0000","last_became_peered":"2024-04-12T19:50:23.685765+0000","last_unstale":"2024-04-12T19:50:25.675846+0000","last_undegraded":"2024-04-12T19:50:25.675846+0000","last_fullsized":"2024-04-12T19:50:25.675846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:04:17.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688457+0000","last_change":"2024-04-12T19:50:24.682664+0000","last_active":"2024-04-12T19:50:25.688457+0000","last_peered":"2024-04-12T19:50:25.688457+0000","last_clean":"2024-04-12T19:50:25.688457+0000","last_became_active":"2024-04-12T19:50:24.682385+0000","last_became_peered":"2024-04-12T19:50:24.682385+0000","last_unstale":"2024-04-12T19:50:25.688457+0000","last_undegraded":"2024-04-12T19:50:25.688457+0000","last_fullsized":"2024-04-12T19:50:25.688457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.701218+0000","last_change":"2024-04-12T19:50:23.689886+0000","last_active":"2024-04-12T19:50:24.701218+0000","last_peered":"2024-04-12T19:50:24.701218+0000","last_clean":"2024-04-12T19:50:24.701218+0000","last_became_active":"2024-04-12T19:50:23.689536+0000","last_became_peered":"2024-04-12T19:50:23.689536+0000","last_unstale":"2024-04-12T19:50:24.701218+0000","last_undegraded":"2024-04-12T19:50:24.701218+0000","last_fullsized":"2024-04-12T19:50:24.701218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:30.521133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675832+0000","last_change":"2024-04-12T19:50:24.689487+0000","last_active":"2024-04-12T19:50:25.675832+0000","last_peered":"2024-04-12T19:50:25.675832+0000","last_clean":"2024-04-12T19:50:25.675832+0000","last_became_active":"2024-04-12T19:50:24.687365+0000","last_became_peered":"2024-04-12T19:50:24.687365+0000","last_unstale":"2024-04-12T19:50:25.675832+0000","last_undegraded":"2024-04-12T19:50:25.675832+0000","last_fullsized":"2024-04-12T19:50:25.675832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184932+0000","last_change":"2024-04-12T19:50:23.682121+0000","last_active":"2024-04-12T19:50:26.184932+0000","last_peered":"2024-04-12T19:50:26.184932+0000","last_clean":"2024-04-12T19:50:26.184932+0000","last_became_active":"2024-04-12T19:50:23.681803+0000","last_became_peered":"2024-04-12T19:50:23.681803+0000","last_unstale":"2024-04-12T19:50:26.184932+0000","last_undegraded":"2024-04-12T19:50:26.184932+0000","last_fullsized":"2024-04-12T19:50:26.184932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184717+0000","last_change":"2024-04-12T19:50:24.685212+0000","last_active":"2024-04-12T19:50:26.184717+0000","last_peered":"2024-04-12T19:50:26.184717+0000","last_clean":"2024-04-12T19:50:26.184717+0000","last_became_active":"2024-04-12T19:50:24.684863+0000","last_became_peered":"2024-04-12T19:50:24.684863+0000","last_unstale":"2024-04-12T19:50:26.184717+0000","last_undegraded":"2024-04-12T19:50:26.184717+0000","last_fullsized":"2024-04-12T19:50:26.184717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:36:27.562288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687944+0000","last_change":"2024-04-12T19:50:23.673596+0000","last_active":"2024-04-12T19:50:25.687944+0000","last_peered":"2024-04-12T19:50:25.687944+0000","last_clean":"2024-04-12T19:50:25.687944+0000","last_became_active":"2024-04-12T19:50:23.672686+0000","last_became_peered":"2024-04-12T19:50:23.672686+0000","last_unstale":"2024-04-12T19:50:25.687944+0000","last_undegraded":"2024-04-12T19:50:25.687944+0000","last_fullsized":"2024-04-12T19:50:25.687944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672719+0000","last_change":"2024-04-12T19:50:24.693699+0000","last_active":"2024-04-12T19:50:25.672719+0000","last_peered":"2024-04-12T19:50:25.672719+0000","last_clean":"2024-04-12T19:50:25.672719+0000","last_became_active":"2024-04-12T19:50:24.693376+0000","last_became_peered":"2024-04-12T19:50:24.693376+0000","last_unstale":"2024-04-12T19:50:25.672719+0000","last_undegraded":"2024-04-12T19:50:25.672719+0000","last_fullsized":"2024-04-12T19:50:25.672719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682161+0000","last_change":"2024-04-12T19:50:23.682371+0000","last_active":"2024-04-12T19:50:25.682161+0000","last_peered":"2024-04-12T19:50:25.682161+0000","last_clean":"2024-04-12T19:50:25.682161+0000","last_became_active":"2024-04-12T19:50:23.681683+0000","last_became_peered":"2024-04-12T19:50:23.681683+0000","last_unstale":"2024-04-12T19:50:25.682161+0000","last_undegraded":"2024-04-12T19:50:25.682161+0000","last_fullsized":"2024-04-12T19:50:25.682161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184368+0000","last_change":"2024-04-12T19:50:24.673851+0000","last_active":"2024-04-12T19:50:26.184368+0000","last_peered":"2024-04-12T19:50:26.184368+0000","last_clean":"2024-04-12T19:50:26.184368+0000","last_became_active":"2024-04-12T19:50:24.673551+0000","last_became_peered":"2024-04-12T19:50:24.673551+0000","last_unstale":"2024-04-12T19:50:26.184368+0000","last_undegraded":"2024-04-12T19:50:26.184368+0000","last_fullsized":"2024-04-12T19:50:26.184368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:29:34.148711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689481+0000","last_change":"2024-04-12T19:50:23.691929+0000","last_active":"2024-04-12T19:50:25.689481+0000","last_peered":"2024-04-12T19:50:25.689481+0000","last_clean":"2024-04-12T19:50:25.689481+0000","last_became_active":"2024-04-12T19:50:23.684811+0000","last_became_peered":"2024-04-12T19:50:23.684811+0000","last_unstale":"2024-04-12T19:50:25.689481+0000","last_undegraded":"2024-04-12T19:50:25.689481+0000","last_fullsized":"2024-04-12T19:50:25.689481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676370+0000","last_change":"2024-04-12T19:50:24.691614+0000","last_active":"2024-04-12T19:50:25.676370+0000","last_peered":"2024-04-12T19:50:25.676370+0000","last_clean":"2024-04-12T19:50:25.676370+0000","last_became_active":"2024-04-12T19:50:24.686194+0000","last_became_peered":"2024-04-12T19:50:24.686194+0000","last_unstale":"2024-04-12T19:50:25.676370+0000","last_undegraded":"2024-04-12T19:50:25.676370+0000","last_fullsized":"2024-04-12T19:50:25.676370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683033+0000","last_change":"2024-04-12T19:50:23.685902+0000","last_active":"2024-04-12T19:50:25.683033+0000","last_peered":"2024-04-12T19:50:25.683033+0000","last_clean":"2024-04-12T19:50:25.683033+0000","last_became_active":"2024-04-12T19:50:23.685614+0000","last_became_peered":"2024-04-12T19:50:23.685614+0000","last_unstale":"2024-04-12T19:50:25.683033+0000","last_undegraded":"2024-04-12T19:50:25.683033+0000","last_fullsized":"2024-04-12T19:50:25.683033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689356+0000","last_change":"2024-04-12T19:50:24.674299+0000","last_active":"2024-04-12T19:50:25.689356+0000","last_peered":"2024-04-12T19:50:25.689356+0000","last_clean":"2024-04-12T19:50:25.689356+0000","last_became_active":"2024-04-12T19:50:24.672616+0000","last_became_peered":"2024-04-12T19:50:24.672616+0000","last_unstale":"2024-04-12T19:50:25.689356+0000","last_undegraded":"2024-04-12T19:50:25.689356+0000","last_fullsized":"2024-04-12T19:50:25.689356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194553+0000","last_change":"2024-04-12T19:50:23.685829+0000","last_active":"2024-04-12T19:50:26.194553+0000","last_peered":"2024-04-12T19:50:26.194553+0000","last_clean":"2024-04-12T19:50:26.194553+0000","last_became_active":"2024-04-12T19:50:23.684944+0000","last_became_peered":"2024-04-12T19:50:23.684944+0000","last_unstale":"2024-04-12T19:50:26.194553+0000","last_undegraded":"2024-04-12T19:50:26.194553+0000","last_fullsized":"2024-04-12T19:50:26.194553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688739+0000","last_change":"2024-04-12T19:50:24.680192+0000","last_active":"2024-04-12T19:50:25.688739+0000","last_peered":"2024-04-12T19:50:25.688739+0000","last_clean":"2024-04-12T19:50:25.688739+0000","last_became_active":"2024-04-12T19:50:24.679916+0000","last_became_peered":"2024-04-12T19:50:24.679916+0000","last_unstale":"2024-04-12T19:50:25.688739+0000","last_undegraded":"2024-04-12T19:50:25.688739+0000","last_fullsized":"2024-04-12T19:50:25.688739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676558+0000","last_change":"2024-04-12T19:50:23.693367+0000","last_active":"2024-04-12T19:50:25.676558+0000","last_peered":"2024-04-12T19:50:25.676558+0000","last_clean":"2024-04-12T19:50:25.676558+0000","last_became_active":"2024-04-12T19:50:23.693036+0000","last_became_peered":"2024-04-12T19:50:23.693036+0000","last_unstale":"2024-04-12T19:50:25.676558+0000","last_undegraded":"2024-04-12T19:50:25.676558+0000","last_fullsized":"2024-04-12T19:50:25.676558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185470+0000","last_change":"2024-04-12T19:50:24.674923+0000","last_active":"2024-04-12T19:50:26.185470+0000","last_peered":"2024-04-12T19:50:26.185470+0000","last_clean":"2024-04-12T19:50:26.185470+0000","last_became_active":"2024-04-12T19:50:24.674524+0000","last_became_peered":"2024-04-12T19:50:24.674524+0000","last_unstale":"2024-04-12T19:50:26.185470+0000","last_undegraded":"2024-04-12T19:50:26.185470+0000","last_fullsized":"2024-04-12T19:50:26.185470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:56.439740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672856+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:50:25.672856+0000","last_peered":"2024-04-12T19:50:25.672856+0000","last_clean":"2024-04-12T19:50:25.672856+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:50:25.672856+0000","last_undegraded":"2024-04-12T19:50:25.672856+0000","last_fullsized":"2024-04-12T19:50:25.672856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184860+0000","last_change":"2024-04-12T19:50:23.676923+0000","last_active":"2024-04-12T19:50:26.184860+0000","last_peered":"2024-04-12T19:50:26.184860+0000","last_clean":"2024-04-12T19:50:26.184860+0000","last_became_active":"2024-04-12T19:50:23.676667+0000","last_became_peered":"2024-04-12T19:50:23.676667+0000","last_unstale":"2024-04-12T19:50:26.184860+0000","last_undegraded":"2024-04-12T19:50:26.184860+0000","last_fullsized":"2024-04-12T19:50:26.184860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:29.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682938+0000","last_change":"2024-04-12T19:50:24.673694+0000","last_active":"2024-04-12T19:50:25.682938+0000","last_peered":"2024-04-12T19:50:25.682938+0000","last_clean":"2024-04-12T19:50:25.682938+0000","last_became_active":"2024-04-12T19:50:24.673340+0000","last_became_peered":"2024-04-12T19:50:24.673340+0000","last_unstale":"2024-04-12T19:50:25.682938+0000","last_undegraded":"2024-04-12T19:50:25.682938+0000","last_fullsized":"2024-04-12T19:50:25.682938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184624+0000","last_change":"2024-04-12T19:50:23.690110+0000","last_active":"2024-04-12T19:50:26.184624+0000","last_peered":"2024-04-12T19:50:26.184624+0000","last_clean":"2024-04-12T19:50:26.184624+0000","last_became_active":"2024-04-12T19:50:23.689677+0000","last_became_peered":"2024-04-12T19:50:23.689677+0000","last_unstale":"2024-04-12T19:50:26.184624+0000","last_undegraded":"2024-04-12T19:50:26.184624+0000","last_fullsized":"2024-04-12T19:50:26.184624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:02.364998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676043+0000","last_change":"2024-04-12T19:50:24.675131+0000","last_active":"2024-04-12T19:50:25.676043+0000","last_peered":"2024-04-12T19:50:25.676043+0000","last_clean":"2024-04-12T19:50:25.676043+0000","last_became_active":"2024-04-12T19:50:24.674486+0000","last_became_peered":"2024-04-12T19:50:24.674486+0000","last_unstale":"2024-04-12T19:50:25.676043+0000","last_undegraded":"2024-04-12T19:50:25.676043+0000","last_fullsized":"2024-04-12T19:50:25.676043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:40.879093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.663820+0000","last_change":"2024-04-12T19:50:23.690325+0000","last_active":"2024-04-12T19:50:24.663820+0000","last_peered":"2024-04-12T19:50:24.663820+0000","last_clean":"2024-04-12T19:50:24.663820+0000","last_became_active":"2024-04-12T19:50:23.688951+0000","last_became_peered":"2024-04-12T19:50:23.688951+0000","last_unstale":"2024-04-12T19:50:24.663820+0000","last_undegraded":"2024-04-12T19:50:24.663820+0000","last_fullsized":"2024-04-12T19:50:24.663820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:49:41.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193423+0000","last_change":"2024-04-12T19:50:24.698245+0000","last_active":"2024-04-12T19:50:26.193423+0000","last_peered":"2024-04-12T19:50:26.193423+0000","last_clean":"2024-04-12T19:50:26.193423+0000","last_became_active":"2024-04-12T19:50:24.697937+0000","last_became_peered":"2024-04-12T19:50:24.697937+0000","last_unstale":"2024-04-12T19:50:26.193423+0000","last_undegraded":"2024-04-12T19:50:26.193423+0000","last_fullsized":"2024-04-12T19:50:26.193423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185039+0000","last_change":"2024-04-12T19:50:23.689007+0000","last_active":"2024-04-12T19:50:26.185039+0000","last_peered":"2024-04-12T19:50:26.185039+0000","last_clean":"2024-04-12T19:50:26.185039+0000","last_became_active":"2024-04-12T19:50:23.688654+0000","last_became_peered":"2024-04-12T19:50:23.688654+0000","last_unstale":"2024-04-12T19:50:26.185039+0000","last_undegraded":"2024-04-12T19:50:26.185039+0000","last_fullsized":"2024-04-12T19:50:26.185039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:24.585940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688821+0000","last_change":"2024-04-12T19:50:24.680562+0000","last_active":"2024-04-12T19:50:25.688821+0000","last_peered":"2024-04-12T19:50:25.688821+0000","last_clean":"2024-04-12T19:50:25.688821+0000","last_became_active":"2024-04-12T19:50:24.680276+0000","last_became_peered":"2024-04-12T19:50:24.680276+0000","last_unstale":"2024-04-12T19:50:25.688821+0000","last_undegraded":"2024-04-12T19:50:25.688821+0000","last_fullsized":"2024-04-12T19:50:25.688821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193774+0000","last_change":"2024-04-12T19:50:23.685407+0000","last_active":"2024-04-12T19:50:26.193774+0000","last_peered":"2024-04-12T19:50:26.193774+0000","last_clean":"2024-04-12T19:50:26.193774+0000","last_became_active":"2024-04-12T19:50:23.684661+0000","last_became_peered":"2024-04-12T19:50:23.684661+0000","last_unstale":"2024-04-12T19:50:26.193774+0000","last_undegraded":"2024-04-12T19:50:26.193774+0000","last_fullsized":"2024-04-12T19:50:26.193774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675625+0000","last_change":"2024-04-12T19:50:24.686111+0000","last_active":"2024-04-12T19:50:25.675625+0000","last_peered":"2024-04-12T19:50:25.675625+0000","last_clean":"2024-04-12T19:50:25.675625+0000","last_became_active":"2024-04-12T19:50:24.685534+0000","last_became_peered":"2024-04-12T19:50:24.685534+0000","last_unstale":"2024-04-12T19:50:25.675625+0000","last_undegraded":"2024-04-12T19:50:25.675625+0000","last_fullsized":"2024-04-12T19:50:25.675625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185739+0000","last_change":"2024-04-12T19:50:23.676900+0000","last_active":"2024-04-12T19:50:26.185739+0000","last_peered":"2024-04-12T19:50:26.185739+0000","last_clean":"2024-04-12T19:50:26.185739+0000","last_became_active":"2024-04-12T19:50:23.676612+0000","last_became_peered":"2024-04-12T19:50:23.676612+0000","last_unstale":"2024-04-12T19:50:26.185739+0000","last_undegraded":"2024-04-12T19:50:26.185739+0000","last_fullsized":"2024-04-12T19:50:26.185739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:39.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676360+0000","last_change":"2024-04-12T19:50:24.695675+0000","last_active":"2024-04-12T19:50:25.676360+0000","last_peered":"2024-04-12T19:50:25.676360+0000","last_clean":"2024-04-12T19:50:25.676360+0000","last_became_active":"2024-04-12T19:50:24.695245+0000","last_became_peered":"2024-04-12T19:50:24.695245+0000","last_unstale":"2024-04-12T19:50:25.676360+0000","last_undegraded":"2024-04-12T19:50:25.676360+0000","last_fullsized":"2024-04-12T19:50:25.676360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.686136+0000","last_change":"2024-04-12T19:50:23.681572+0000","last_active":"2024-04-12T19:50:25.686136+0000","last_peered":"2024-04-12T19:50:25.686136+0000","last_clean":"2024-04-12T19:50:25.686136+0000","last_became_active":"2024-04-12T19:50:23.679612+0000","last_became_peered":"2024-04-12T19:50:23.679612+0000","last_unstale":"2024-04-12T19:50:25.686136+0000","last_undegraded":"2024-04-12T19:50:25.686136+0000","last_fullsized":"2024-04-12T19:50:25.686136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682950+0000","last_change":"2024-04-12T19:50:24.686596+0000","last_active":"2024-04-12T19:50:25.682950+0000","last_peered":"2024-04-12T19:50:25.682950+0000","last_clean":"2024-04-12T19:50:25.682950+0000","last_became_active":"2024-04-12T19:50:24.685023+0000","last_became_peered":"2024-04-12T19:50:24.685023+0000","last_unstale":"2024-04-12T19:50:25.682950+0000","last_undegraded":"2024-04-12T19:50:25.682950+0000","last_fullsized":"2024-04-12T19:50:25.682950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185887+0000","last_change":"2024-04-12T19:50:23.684680+0000","last_active":"2024-04-12T19:50:26.185887+0000","last_peered":"2024-04-12T19:50:26.185887+0000","last_clean":"2024-04-12T19:50:26.185887+0000","last_became_active":"2024-04-12T19:50:23.684298+0000","last_became_peered":"2024-04-12T19:50:23.684298+0000","last_unstale":"2024-04-12T19:50:26.185887+0000","last_undegraded":"2024-04-12T19:50:26.185887+0000","last_fullsized":"2024-04-12T19:50:26.185887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:06.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683078+0000","last_change":"2024-04-12T19:50:24.676462+0000","last_active":"2024-04-12T19:50:25.683078+0000","last_peered":"2024-04-12T19:50:25.683078+0000","last_clean":"2024-04-12T19:50:25.683078+0000","last_became_active":"2024-04-12T19:50:24.676182+0000","last_became_peered":"2024-04-12T19:50:24.676182+0000","last_unstale":"2024-04-12T19:50:25.683078+0000","last_undegraded":"2024-04-12T19:50:25.683078+0000","last_fullsized":"2024-04-12T19:50:25.683078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687692+0000","last_change":"2024-04-12T19:50:23.684797+0000","last_active":"2024-04-12T19:50:25.687692+0000","last_peered":"2024-04-12T19:50:25.687692+0000","last_clean":"2024-04-12T19:50:25.687692+0000","last_became_active":"2024-04-12T19:50:23.678108+0000","last_became_peered":"2024-04-12T19:50:23.678108+0000","last_unstale":"2024-04-12T19:50:25.687692+0000","last_undegraded":"2024-04-12T19:50:25.687692+0000","last_fullsized":"2024-04-12T19:50:25.687692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184277+0000","last_change":"2024-04-12T19:50:24.681150+0000","last_active":"2024-04-12T19:50:26.184277+0000","last_peered":"2024-04-12T19:50:26.184277+0000","last_clean":"2024-04-12T19:50:26.184277+0000","last_became_active":"2024-04-12T19:50:24.680857+0000","last_became_peered":"2024-04-12T19:50:24.680857+0000","last_unstale":"2024-04-12T19:50:26.184277+0000","last_undegraded":"2024-04-12T19:50:26.184277+0000","last_fullsized":"2024-04-12T19:50:26.184277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682575+0000","last_change":"2024-04-12T19:50:23.691210+0000","last_active":"2024-04-12T19:50:25.682575+0000","last_peered":"2024-04-12T19:50:25.682575+0000","last_clean":"2024-04-12T19:50:25.682575+0000","last_became_active":"2024-04-12T19:50:23.690943+0000","last_became_peered":"2024-04-12T19:50:23.690943+0000","last_unstale":"2024-04-12T19:50:25.682575+0000","last_undegraded":"2024-04-12T19:50:25.682575+0000","last_fullsized":"2024-04-12T19:50:25.682575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682186+0000","last_change":"2024-04-12T19:50:24.667678+0000","last_active":"2024-04-12T19:50:25.682186+0000","last_peered":"2024-04-12T19:50:25.682186+0000","last_clean":"2024-04-12T19:50:25.682186+0000","last_became_active":"2024-04-12T19:50:24.667407+0000","last_became_peered":"2024-04-12T19:50:24.667407+0000","last_unstale":"2024-04-12T19:50:25.682186+0000","last_undegraded":"2024-04-12T19:50:25.682186+0000","last_fullsized":"2024-04-12T19:50:25.682186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676272+0000","last_change":"2024-04-12T19:50:23.690317+0000","last_active":"2024-04-12T19:50:25.676272+0000","last_peered":"2024-04-12T19:50:25.676272+0000","last_clean":"2024-04-12T19:50:25.676272+0000","last_became_active":"2024-04-12T19:50:23.689037+0000","last_became_peered":"2024-04-12T19:50:23.689037+0000","last_unstale":"2024-04-12T19:50:25.676272+0000","last_undegraded":"2024-04-12T19:50:25.676272+0000","last_fullsized":"2024-04-12T19:50:25.676272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-12T19:50:24.695833+0000","last_change":"2024-04-12T19:50:24.681363+0000","last_active":"2024-04-12T19:50:24.695833+0000","last_peered":"2024-04-12T19:50:24.695833+0000","last_clean":"2024-04-12T19:50:24.695833+0000","last_became_active":"2024-04-12T19:50:24.681100+0000","last_became_peered":"2024-04-12T19:50:24.681100+0000","last_unstale":"2024-04-12T19:50:24.695833+0000","last_undegraded":"2024-04-12T19:50:24.695833+0000","last_fullsized":"2024-04-12T19:50:24.695833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:41:07.324095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184678+0000","last_change":"2024-04-12T19:50:23.679805+0000","last_active":"2024-04-12T19:50:26.184678+0000","last_peered":"2024-04-12T19:50:26.184678+0000","last_clean":"2024-04-12T19:50:26.184678+0000","last_became_active":"2024-04-12T19:50:23.679501+0000","last_became_peered":"2024-04-12T19:50:23.679501+0000","last_unstale":"2024-04-12T19:50:26.184678+0000","last_undegraded":"2024-04-12T19:50:26.184678+0000","last_fullsized":"2024-04-12T19:50:26.184678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:44.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184641+0000","last_change":"2024-04-12T19:50:24.678712+0000","last_active":"2024-04-12T19:50:26.184641+0000","last_peered":"2024-04-12T19:50:26.184641+0000","last_clean":"2024-04-12T19:50:26.184641+0000","last_became_active":"2024-04-12T19:50:24.677584+0000","last_became_peered":"2024-04-12T19:50:24.677584+0000","last_unstale":"2024-04-12T19:50:26.184641+0000","last_undegraded":"2024-04-12T19:50:26.184641+0000","last_fullsized":"2024-04-12T19:50:26.184641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:27:08.109541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185338+0000","last_change":"2024-04-12T19:50:24.698869+0000","last_active":"2024-04-12T19:50:26.185338+0000","last_peered":"2024-04-12T19:50:26.185338+0000","last_clean":"2024-04-12T19:50:26.185338+0000","last_became_active":"2024-04-12T19:50:24.698605+0000","last_became_peered":"2024-04-12T19:50:24.698605+0000","last_unstale":"2024-04-12T19:50:26.185338+0000","last_undegraded":"2024-04-12T19:50:26.185338+0000","last_fullsized":"2024-04-12T19:50:26.185338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:26:54.420503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688867+0000","last_change":"2024-04-12T19:50:23.678658+0000","last_active":"2024-04-12T19:50:25.688867+0000","last_peered":"2024-04-12T19:50:25.688867+0000","last_clean":"2024-04-12T19:50:25.688867+0000","last_became_active":"2024-04-12T19:50:23.676520+0000","last_became_peered":"2024-04-12T19:50:23.676520+0000","last_unstale":"2024-04-12T19:50:25.688867+0000","last_undegraded":"2024-04-12T19:50:25.688867+0000","last_fullsized":"2024-04-12T19:50:25.688867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673494+0000","last_change":"2024-04-12T19:50:24.684296+0000","last_active":"2024-04-12T19:50:25.673494+0000","last_peered":"2024-04-12T19:50:25.673494+0000","last_clean":"2024-04-12T19:50:25.673494+0000","last_became_active":"2024-04-12T19:50:24.683867+0000","last_became_peered":"2024-04-12T19:50:24.683867+0000","last_unstale":"2024-04-12T19:50:25.673494+0000","last_undegraded":"2024-04-12T19:50:25.673494+0000","last_fullsized":"2024-04-12T19:50:25.673494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688543+0000","last_change":"2024-04-12T19:50:23.680894+0000","last_active":"2024-04-12T19:50:25.688543+0000","last_peered":"2024-04-12T19:50:25.688543+0000","last_clean":"2024-04-12T19:50:25.688543+0000","last_became_active":"2024-04-12T19:50:23.679663+0000","last_became_peered":"2024-04-12T19:50:23.679663+0000","last_unstale":"2024-04-12T19:50:25.688543+0000","last_undegraded":"2024-04-12T19:50:25.688543+0000","last_fullsized":"2024-04-12T19:50:25.688543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194138+0000","last_change":"2024-04-12T19:50:24.695055+0000","last_active":"2024-04-12T19:50:26.194138+0000","last_peered":"2024-04-12T19:50:26.194138+0000","last_clean":"2024-04-12T19:50:26.194138+0000","last_became_active":"2024-04-12T19:50:24.694565+0000","last_became_peered":"2024-04-12T19:50:24.694565+0000","last_unstale":"2024-04-12T19:50:26.194138+0000","last_undegraded":"2024-04-12T19:50:26.194138+0000","last_fullsized":"2024-04-12T19:50:26.194138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194545+0000","last_change":"2024-04-12T19:50:23.684953+0000","last_active":"2024-04-12T19:50:26.194545+0000","last_peered":"2024-04-12T19:50:26.194545+0000","last_clean":"2024-04-12T19:50:26.194545+0000","last_became_active":"2024-04-12T19:50:23.684555+0000","last_became_peered":"2024-04-12T19:50:23.684555+0000","last_unstale":"2024-04-12T19:50:26.194545+0000","last_undegraded":"2024-04-12T19:50:26.194545+0000","last_fullsized":"2024-04-12T19:50:26.194545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194574+0000","last_change":"2024-04-12T19:50:24.701879+0000","last_active":"2024-04-12T19:50:26.194574+0000","last_peered":"2024-04-12T19:50:26.194574+0000","last_clean":"2024-04-12T19:50:26.194574+0000","last_became_active":"2024-04-12T19:50:24.701573+0000","last_became_peered":"2024-04-12T19:50:24.701573+0000","last_unstale":"2024-04-12T19:50:26.194574+0000","last_undegraded":"2024-04-12T19:50:26.194574+0000","last_fullsized":"2024-04-12T19:50:26.194574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683114+0000","last_change":"2024-04-12T19:50:23.673049+0000","last_active":"2024-04-12T19:50:25.683114+0000","last_peered":"2024-04-12T19:50:25.683114+0000","last_clean":"2024-04-12T19:50:25.683114+0000","last_became_active":"2024-04-12T19:50:23.672610+0000","last_became_peered":"2024-04-12T19:50:23.672610+0000","last_unstale":"2024-04-12T19:50:25.683114+0000","last_undegraded":"2024-04-12T19:50:25.683114+0000","last_fullsized":"2024-04-12T19:50:25.683114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":70566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:50:29.727 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-12T19:50:29.727 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-12T19:50:29.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-12T19:50:29.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-12T19:50:29.728 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-12T19:50:29.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.988+0000 7fdebb000640 1 Processor -- start 2024-04-12T19:50:29.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.988+0000 7fdebb000640 1 -- start start 2024-04-12T19:50:29.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdebb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fdeb406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdebb000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb415dbb0 con 0x7fdeb40cee00 2024-04-12T19:50:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdebb000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb415d8d0 con 0x7fdeb4072660 2024-04-12T19:50:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdebb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb406f4f0 con 0x7fdeb415d2f0 2024-04-12T19:50:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdeb9c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdeb40cee00 0x7fdeb4071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43206/0 (socket says 172.21.15.32:43206) 2024-04-12T19:50:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.990+0000 7fdeb9c00640 1 -- 172.21.15.32:0/2195158157 learned_addr learned my addr 172.21.15.32:0/2195158157 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551163880 0 0) 0x7fdeb415dbb0 con 0x7fdeb40cee00 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde9c003520 con 0x7fdeb40cee00 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeba600640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fdeb406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 811971630 0 0) 0x7fde9c003520 con 0x7fdeb40cee00 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 msgr2=0x7fdeb406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fdeb406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 >> v1:172.21.15.169:6789/0 conn(0x7fdeb4072660 legacy=0x7fdeb415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb406dde0 con 0x7fdeb40cee00 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fdea8001670 con 0x7fdeb40cee00 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 >> v1:172.21.15.32:6789/0 conn(0x7fdeb40cee00 legacy=0x7fdeb4071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fde9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.991+0000 7fdeb3e00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9c004010 0x7fde9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb3e00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fde9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c00bb50 con 0x7fde9c004d40 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c00bd00 con 0x7fde9c004010 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c00bf00 con 0x7fdeb415d2f0 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fde9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb9c00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fde9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb9200640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9c004010 0x7fde9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 -- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 msgr2=0x7fde9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fde9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 -- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9c004010 msgr2=0x7fde9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9c004010 0x7fde9c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeba600640 1 -- 172.21.15.32:0/2195158157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c003520 con 0x7fde9c004d40 2024-04-12T19:50:29.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.992+0000 7fdeb9c00640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb415d2f0 0x7fde9c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:29.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.993+0000 7fdeba600640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fde9c0051c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007360 tx=0x7fde9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0e650505de84aa server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:29.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac039070 con 0x7fde9c004d40 2024-04-12T19:50:29.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac02ccb0 con 0x7fde9c004d40 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdeb3e00640 1 -- 172.21.15.32:0/2195158157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac009be0 con 0x7fde9c004d40 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 -- 172.21.15.32:0/2195158157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 msgr2=0x7fde9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 --2- 172.21.15.32:0/2195158157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fde9c0051c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007360 tx=0x7fde9c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdeba600640 1 -- 172.21.15.32:0/2195158157 reap_dead start 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 -- 172.21.15.32:0/2195158157 shutdown_connections 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 -- 172.21.15.32:0/2195158157 >> 172.21.15.32:0/2195158157 conn(0x7fdeb406b930 msgr2=0x7fdeb406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 -- 172.21.15.32:0/2195158157 shutdown_connections 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.994+0000 7fdebb000640 1 -- 172.21.15.32:0/2195158157 wait complete. 2024-04-12T19:50:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.996+0000 7fdebb000640 1 Processor -- start 2024-04-12T19:50:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.996+0000 7fdebb000640 1 -- start start 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb414a2b0 0x7fdeb414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb414be50 0x7fdeb40c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb406e8c0 con 0x7fde9c004d40 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb406e3a0 con 0x7fdeb414a2b0 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdebb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb406ea40 con 0x7fdeb414be50 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.997+0000 7fdeb9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60960/0 (socket says 172.21.15.32:60960) 2024-04-12T19:50:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 -- 172.21.15.32:0/2528408149 learned_addr learned my addr 172.21.15.32:0/2528408149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeba600640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb414be50 0x7fdeb40c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9200640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb414a2b0 0x7fdeb414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 -- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb414be50 msgr2=0x7fdeb40c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb414be50 0x7fdeb40c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 -- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb414a2b0 msgr2=0x7fdeb414b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb414a2b0 0x7fdeb414b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 -- 172.21.15.32:0/2528408149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb414a870 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.998+0000 7fdeb9c00640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdea800d910 tx=0x7fdea800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.999+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8019070 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.999+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea80074e0 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.999+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80049e0 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.999+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb414ab50 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:29.999+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb414b110 con 0x7fde9c004d40 2024-04-12T19:50:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.000+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb40755c0 con 0x7fde9c004d40 2024-04-12T19:50:30.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.001+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdea8004b80 con 0x7fde9c004d40 2024-04-12T19:50:30.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.002+0000 7fdeb1600640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 0x7fde9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.002+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdea808b710 con 0x7fde9c004d40 2024-04-12T19:50:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.002+0000 7fdeb9200640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 0x7fde9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.003+0000 7fdeb9200640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 0x7fde9406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40c5710 tx=0x7fdea0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.006+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdea8007690 con 0x7fde9c004d40 2024-04-12T19:50:30.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.260+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdeb415d030 con 0x7fde9c004d40 2024-04-12T19:50:30.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.269+0000 7fdeb1600640 1 -- 172.21.15.32:0/2528408149 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174135 (secure 0 0 0) 0x7fdea8059b60 con 0x7fde9c004d40 2024-04-12T19:50:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.272+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 msgr2=0x7fde9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.272+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 0x7fde9406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40c5710 tx=0x7fdea0009040 comp rx=0 tx=0).stop 2024-04-12T19:50:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 msgr2=0x7fdeb4149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdea800d910 tx=0x7fdea800ddd0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 shutdown_connections 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fde9406c8f0 0x7fde9406edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb414be50 0x7fdeb40c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb414a2b0 0x7fdeb414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 --2- 172.21.15.32:0/2528408149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9c004d40 0x7fdeb4149b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 >> 172.21.15.32:0/2528408149 conn(0x7fdeb406b930 msgr2=0x7fdeb415c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 shutdown_connections 2024-04-12T19:50:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.273+0000 7fdebb000640 1 -- 172.21.15.32:0/2528408149 wait complete. 2024-04-12T19:50:30.290 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1274114417},{"type":"v1","addr":"172.21.15.32:6833","nonce":1274114417}]},"active_addr":"172.21.15.32:6833/1274114417","active_change":"2024-04-12T19:50:03.602064+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":597995495}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3759860358}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":614977979}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4125102973}]}]} 2024-04-12T19:50:30.294 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-12T19:50:30.294 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-12T19:50:30.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.515+0000 7f632b200640 1 Processor -- start 2024-04-12T19:50:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.515+0000 7f632b200640 1 -- start start 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.516+0000 7f632b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f632c09b150 0x7f632c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.516+0000 7f632b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632c06ca40 con 0x7f632c0b6ea0 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.516+0000 7f632b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632c06cce0 con 0x7f632c15e7e0 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.516+0000 7f632b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f632c0b72f0 con 0x7f632c09b150 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.517+0000 7f6329400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f632c0b6ea0 0x7f632c09aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43218/0 (socket says 172.21.15.32:43218) 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.517+0000 7f6329400640 1 -- 172.21.15.32:0/3238462154 learned_addr learned my addr 172.21.15.32:0/3238462154 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.517+0000 7f632a800640 1 --2- 172.21.15.32:0/3238462154 >> v2:172.21.15.169:3300/0 conn(0x7f632c09b150 0x7f632c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.517+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814117335 0 0) 0x7f632c06ca40 con 0x7f632c0b6ea0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.517+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f630c003520 con 0x7f632c0b6ea0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2335620946 0 0) 0x7f630c003520 con 0x7f632c0b6ea0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 >> v2:172.21.15.169:3300/0 conn(0x7f632c09b150 msgr2=0x7f632c0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 --2- 172.21.15.32:0/3238462154 >> v2:172.21.15.169:3300/0 conn(0x7f632c09b150 0x7f632c0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 >> v1:172.21.15.169:6789/0 conn(0x7f632c15e7e0 legacy=0x7f632c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c06cfc0 con 0x7f632c0b6ea0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f63140019f0 con 0x7f632c0b6ea0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 >> v1:172.21.15.32:6789/0 conn(0x7f632c0b6ea0 legacy=0x7f632c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c0039b0 0x7f630c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c004800 0x7f630c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f630c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f630c00c030 con 0x7f630c0051c0 2024-04-12T19:50:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f630c00c1e0 con 0x7f630c004800 2024-04-12T19:50:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f630c00c3e0 con 0x7f630c0039b0 2024-04-12T19:50:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f632a800640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f630c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6329400640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c004800 0x7f630c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.518+0000 7f6329e00640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c0039b0 0x7f630c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 -- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c0039b0 msgr2=0x7f630c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c0039b0 0x7f630c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 -- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c004800 msgr2=0x7f630c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c004800 0x7f630c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 -- 172.21.15.32:0/3238462154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c003520 con 0x7f630c0051c0 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.519+0000 7f632a800640 1 -- 172.21.15.32:0/3238462154 reap_dead start 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.520+0000 7f632a800640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f630c00b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6324009ac0 tx=0x7f630c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f32db15eb022ac server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.520+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324009690 con 0x7f630c0051c0 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.520+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324004020 con 0x7f630c0051c0 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.520+0000 7f6328a00640 1 -- 172.21.15.32:0/3238462154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324004410 con 0x7f630c0051c0 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 -- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 msgr2=0x7f630c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f630c00b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6324009ac0 tx=0x7f630c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 -- 172.21.15.32:0/3238462154 shutdown_connections 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 --2- 172.21.15.32:0/3238462154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f630c00b9c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 -- 172.21.15.32:0/3238462154 >> 172.21.15.32:0/3238462154 conn(0x7f632c0b40a0 msgr2=0x7f632c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 -- 172.21.15.32:0/3238462154 shutdown_connections 2024-04-12T19:50:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.521+0000 7f632b200640 1 -- 172.21.15.32:0/3238462154 wait complete. 2024-04-12T19:50:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.522+0000 7f632b200640 1 Processor -- start 2024-04-12T19:50:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.523+0000 7f632b200640 1 -- start start 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f632c0cb760 0x7f632c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f632c151d30 0x7f632c13ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f632c06d580 con 0x7f630c0051c0 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f632c06d8f0 con 0x7f632c0cb760 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f632c15ebc0 con 0x7f632c151d30 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60978/0 (socket says 172.21.15.32:60978) 2024-04-12T19:50:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 -- 172.21.15.32:0/2051945284 learned_addr learned my addr 172.21.15.32:0/2051945284 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329400640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f632c0cb760 0x7f632c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 -- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f632c151d30 msgr2=0x7f632c13ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f632a800640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f632c151d30 0x7f632c13ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f632c151d30 0x7f632c13ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 -- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f632c0cb760 msgr2=0x7f632c14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f632c0cb760 0x7f632c14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.524+0000 7f6329e00640 1 -- 172.21.15.32:0/2051945284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c150c90 con 0x7f630c0051c0 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f6329e00640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e500 tx=0x7f631c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c0052e0 con 0x7f630c0051c0 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c018070 con 0x7f630c0051c0 2024-04-12T19:50:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003810 con 0x7f630c0051c0 2024-04-12T19:50:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c150f70 con 0x7f630c0051c0 2024-04-12T19:50:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.525+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c151530 con 0x7f630c0051c0 2024-04-12T19:50:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.526+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c0755c0 con 0x7f630c0051c0 2024-04-12T19:50:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.527+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f631c0039b0 con 0x7f630c0051c0 2024-04-12T19:50:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.528+0000 7f6322000640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 0x7f630806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.528+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f631c08b7b0 con 0x7f630c0051c0 2024-04-12T19:50:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.528+0000 7f6329400640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 0x7f630806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.529+0000 7f6329400640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 0x7f630806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314001580 tx=0x7f63140012b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.533+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f631c059b50 con 0x7f630c0051c0 2024-04-12T19:50:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.734+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f632c09b750 con 0x7f630c0051c0 2024-04-12T19:50:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.737+0000 7f6322000640 1 -- 172.21.15.32:0/2051945284 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7f631c059450 con 0x7f630c0051c0 2024-04-12T19:50:30.737 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:30.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 msgr2=0x7f630806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 0x7f630806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314001580 tx=0x7f63140012b0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 msgr2=0x7f632c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f631c00e500 tx=0x7f631c00e9c0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 shutdown_connections 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f630806c920 0x7f630806ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f632c151d30 0x7f632c13ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f632c0cb760 0x7f632c14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 --2- 172.21.15.32:0/2051945284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f630c0051c0 0x7f632c156010 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 >> 172.21.15.32:0/2051945284 conn(0x7f632c0b40a0 msgr2=0x7f632c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 shutdown_connections 2024-04-12T19:50:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.740+0000 7f632b200640 1 -- 172.21.15.32:0/2051945284 wait complete. 2024-04-12T19:50:30.756 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-12T19:50:30.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.958+0000 7f26fac00640 1 Processor -- start 2024-04-12T19:50:30.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.958+0000 7f26fac00640 1 -- start start 2024-04-12T19:50:30.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26fac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26f4148130 0x7f26f415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26fac00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26f4162380 0x7f26f4071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26fac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f4162940 con 0x7f26f4072490 2024-04-12T19:50:30.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26fac00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f26f4072900 con 0x7f26f4162380 2024-04-12T19:50:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26fac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f26f4148510 con 0x7f26f4148130 2024-04-12T19:50:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26f8e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26f4162380 0x7f26f4071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26f8e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26f4162380 0x7f26f4071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60980/0 (socket says 172.21.15.32:60980) 2024-04-12T19:50:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26f8e00640 1 -- 172.21.15.32:0/3921006520 learned_addr learned my addr 172.21.15.32:0/3921006520 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.960+0000 7f26f9800640 1 --2- 172.21.15.32:0/3921006520 >> v2:172.21.15.169:3300/0 conn(0x7f26f4148130 0x7f26f415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.961+0000 7f26f8e00640 1 -- 172.21.15.32:0/3921006520 >> v2:172.21.15.169:3300/0 conn(0x7f26f4148130 msgr2=0x7f26f415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.961+0000 7f26f8e00640 1 --2- 172.21.15.32:0/3921006520 >> v2:172.21.15.169:3300/0 conn(0x7f26f4148130 0x7f26f415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.961+0000 7f26f8e00640 1 -- 172.21.15.32:0/3921006520 >> v1:172.21.15.169:6789/0 conn(0x7f26f4072490 legacy=0x7f26f406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.961+0000 7f26f8e00640 1 -- 172.21.15.32:0/3921006520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f4162c20 con 0x7f26f4162380 2024-04-12T19:50:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.962+0000 7f26f8e00640 1 --2- 172.21.15.32:0/3921006520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4162380 0x7f26f4071f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26f4073010 tx=0x7f26e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d168ef52bad00e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.962+0000 7f26f3e00640 1 -- 172.21.15.32:0/3921006520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e002e070 con 0x7f26f4162380 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.962+0000 7f26f3e00640 1 -- 172.21.15.32:0/3921006520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e0037070 con 0x7f26f4162380 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.962+0000 7f26f3e00640 1 -- 172.21.15.32:0/3921006520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0031a00 con 0x7f26f4162380 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 -- 172.21.15.32:0/3921006520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4162380 msgr2=0x7f26f4071f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 --2- 172.21.15.32:0/3921006520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4162380 0x7f26f4071f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26f4073010 tx=0x7f26e002b880 comp rx=0 tx=0).stop 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 -- 172.21.15.32:0/3921006520 shutdown_connections 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 --2- 172.21.15.32:0/3921006520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4162380 0x7f26f4071f50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 --2- 172.21.15.32:0/3921006520 >> v2:172.21.15.169:3300/0 conn(0x7f26f4148130 0x7f26f415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 -- 172.21.15.32:0/3921006520 >> 172.21.15.32:0/3921006520 conn(0x7f26f406b930 msgr2=0x7f26f406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 -- 172.21.15.32:0/3921006520 shutdown_connections 2024-04-12T19:50:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.963+0000 7f26fac00640 1 -- 172.21.15.32:0/3921006520 wait complete. 2024-04-12T19:50:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.965+0000 7f26fac00640 1 Processor -- start 2024-04-12T19:50:30.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.965+0000 7f26fac00640 1 -- start start 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26f4072490 0x7f26f413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26f4162380 0x7f26f406dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41631e0 con 0x7f26f4148130 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26f415fd70 con 0x7f26f4162380 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26fac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26f40aeb90 con 0x7f26f4072490 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26f8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26f8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60982/0 (socket says 172.21.15.32:60982) 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.966+0000 7f26f8e00640 1 -- 172.21.15.32:0/1302685127 learned_addr learned my addr 172.21.15.32:0/1302685127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f9800640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26f4072490 0x7f26f413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26fa200640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26f4162380 0x7f26f406dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 -- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26f4072490 msgr2=0x7f26f413ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26f4072490 0x7f26f413ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 -- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26f4162380 msgr2=0x7f26f406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26f4162380 0x7f26f406dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 -- 172.21.15.32:0/1302685127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f40cf700 con 0x7f26f4148130 2024-04-12T19:50:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f8e00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f26e002b8c0 tx=0x7f26e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0031ed0 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e00328a0 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0032c90 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.967+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f40cf9e0 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.968+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f40af250 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.969+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f40755c0 con 0x7f26f4148130 2024-04-12T19:50:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.970+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26e004c060 con 0x7f26f4148130 2024-04-12T19:50:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.970+0000 7f26f1600640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 0x7f26d406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.971+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f26e00ad1b0 con 0x7f26f4148130 2024-04-12T19:50:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.971+0000 7f26f9800640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 0x7f26d406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.971+0000 7f26f9800640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 0x7f26d406ee30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26e8006fd0 tx=0x7f26e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:30.975+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26e0040070 con 0x7f26f4148130 2024-04-12T19:50:31.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.179+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26f406f060 con 0x7f26f4148130 2024-04-12T19:50:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.182+0000 7f26f1600640 1 -- 172.21.15.32:0/1302685127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7f26e007b6c0 con 0x7f26f4148130 2024-04-12T19:50:31.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:31.182 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.184+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 msgr2=0x7f26d406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.184+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 0x7f26d406ee30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26e8006fd0 tx=0x7f26e8008040 comp rx=0 tx=0).stop 2024-04-12T19:50:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.184+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 msgr2=0x7f26f40cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.184+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f26e002b8c0 tx=0x7f26e00046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 shutdown_connections 2024-04-12T19:50:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f26d406c970 0x7f26d406ee30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26f4162380 0x7f26f406dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4148130 0x7f26f40cedb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 --2- 172.21.15.32:0/1302685127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26f4072490 0x7f26f413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 >> 172.21.15.32:0/1302685127 conn(0x7f26f406b930 msgr2=0x7f26f415f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 shutdown_connections 2024-04-12T19:50:31.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.185+0000 7f26fac00640 1 -- 172.21.15.32:0/1302685127 wait complete. 2024-04-12T19:50:31.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-12T19:50:31.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-12T19:50:31.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-12T19:50:31.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-12T19:50:31.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-12T19:50:31.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-12T19:50:31.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-12T19:50:31.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-12T19:50:31.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.519+0000 7fddd7200640 1 Processor -- start 2024-04-12T19:50:31.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.521+0000 7f6c44a00640 1 Processor -- start 2024-04-12T19:50:31.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.520+0000 7ff541a00640 1 Processor -- start 2024-04-12T19:50:31.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.520+0000 7ff541a00640 1 -- start start 2024-04-12T19:50:31.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.520+0000 7fddd7200640 1 -- start start 2024-04-12T19:50:31.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7ff541a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 0x7ff53c16fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7ff541a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff53c06ff00 0x7ff53c070340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7ff541a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff53c1701b0 con 0x7ff53c16b9c0 2024-04-12T19:50:31.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7ff541a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff53c0b3760 con 0x7ff53c06ff00 2024-04-12T19:50:31.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7ff541a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff53c070960 con 0x7ff53c0b4490 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7fddd7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fddd003c0a0 0x7fddd00e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7fddd7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7fddd7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd0054680 con 0x7fddd00e3160 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7fddd7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fddd0007b70 con 0x7fddd003c0a0 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.522+0000 7fddd7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fddd0058810 con 0x7fddd0053cb0 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7fddd5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fddd003c0a0 0x7fddd00e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7fddd6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7f6c44a00640 1 -- start start 2024-04-12T19:50:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7fddd6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:54910/0 (socket says 172.21.15.32:54910) 2024-04-12T19:50:31.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7fddd6800640 1 -- 172.21.15.32:0/3923585795 learned_addr learned my addr 172.21.15.32:0/3923585795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7ff53b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff53c16b9c0 0x7ff53c16be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43228/0 (socket says 172.21.15.32:43228) 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7ff53b400640 1 -- 172.21.15.32:0/3676408714 learned_addr learned my addr 172.21.15.32:0/3676408714 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7ff53aa00640 1 --2- 172.21.15.32:0/3676408714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 0x7ff53c16fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.523+0000 7ff53a000640 1 --2- 172.21.15.32:0/3676408714 >> v2:172.21.15.169:3300/0 conn(0x7ff53c06ff00 0x7ff53c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7ff53aa00640 1 -- 172.21.15.32:0/3676408714 >> v2:172.21.15.169:3300/0 conn(0x7ff53c06ff00 msgr2=0x7ff53c070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7ff53aa00640 1 --2- 172.21.15.32:0/3676408714 >> v2:172.21.15.169:3300/0 conn(0x7ff53c06ff00 0x7ff53c070340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7ff53aa00640 1 -- 172.21.15.32:0/3676408714 >> v1:172.21.15.32:6789/0 conn(0x7ff53c16b9c0 legacy=0x7ff53c16be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7ff53aa00640 1 -- 172.21.15.32:0/3676408714 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c0b39b0 con 0x7ff53c0b4490 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7fddd6800640 1 -- 172.21.15.32:0/3923585795 >> v2:172.21.15.169:3300/0 conn(0x7fddd003c0a0 msgr2=0x7fddd00e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7fddd6800640 1 --2- 172.21.15.32:0/3923585795 >> v2:172.21.15.169:3300/0 conn(0x7fddd003c0a0 0x7fddd00e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7fddd6800640 1 -- 172.21.15.32:0/3923585795 >> v1:172.21.15.169:6789/0 conn(0x7fddd00e3160 legacy=0x7fddd0058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.524+0000 7fddd6800640 1 -- 172.21.15.32:0/3923585795 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd0058af0 con 0x7fddd0053cb0 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f6c44a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3800a430 0x7f6c38055d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7fddd6800640 1 --2- 172.21.15.32:0/3923585795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fddd8074520 tx=0x7fddd80a75b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac8da0ae5a608ad server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f6c3f400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6c380e12e0 0x7f6c3803a140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:55588/0 (socket says 172.21.15.32:55588) 2024-04-12T19:50:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f6c3ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c3800ab10 0x7f6c38009d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43232/0 (socket says 172.21.15.32:43232) 2024-04-12T19:50:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f6c3f400640 1 -- 172.21.15.32:0/3322566945 learned_addr learned my addr 172.21.15.32:0/3322566945 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f2fad200640 1 Processor -- start 2024-04-12T19:50:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c38054ce0 con 0x7f6c3800ab10 2024-04-12T19:50:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7f2fad200640 1 -- start start 2024-04-12T19:50:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c38054fc0 con 0x7f6c380e12e0 2024-04-12T19:50:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c380558f0 con 0x7f6c3800a430 2024-04-12T19:50:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588415402 0 0) 0x7f6c38054ce0 con 0x7f6c3800ab10 2024-04-12T19:50:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c28003520 con 0x7f6c3800ab10 2024-04-12T19:50:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7fddd4a00640 1 -- 172.21.15.32:0/3923585795 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80b3070 con 0x7fddd0053cb0 2024-04-12T19:50:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136475837 0 0) 0x7f6c38054fc0 con 0x7f6c380e12e0 2024-04-12T19:50:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7fddd4a00640 1 -- 172.21.15.32:0/3923585795 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd80ac830 con 0x7fddd0053cb0 2024-04-12T19:50:31.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7fddd4a00640 1 -- 172.21.15.32:0/3923585795 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80acca0 con 0x7fddd0053cb0 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c38054ce0 con 0x7f6c380e12e0 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3908193453 0 0) 0x7f6c28003520 con 0x7f6c3800ab10 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3800a430 msgr2=0x7f6c38055d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c3800a430 0x7f6c38055d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 >> v1:172.21.15.169:6789/0 conn(0x7f6c380e12e0 legacy=0x7f6c3803a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c380552a0 con 0x7f6c3800ab10 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6c34001670 con 0x7f6c3800ab10 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 >> v1:172.21.15.32:6789/0 conn(0x7f6c3800ab10 legacy=0x7f6c38009d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28003b70 0x7f6c28003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c380558f0 con 0x7f6c28003b70 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800c110 con 0x7f6c28005210 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2800c310 con 0x7f6c28004800 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f2fad200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fa815c710 0x7f2fa80b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f2fad200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fa809aa40 0x7f2fa809aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f2fad200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa80b68e0 con 0x7f2fa80b08c0 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f2fad200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa80afbb0 con 0x7f2fa815c710 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.528+0000 7f2fad200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa809b3e0 con 0x7f2fa809aa40 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.526+0000 7ff53aa00640 1 --2- 172.21.15.32:0/3676408714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 0x7ff53c16fc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0b38e0 tx=0x7ff52c02ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6933482ca6c35684 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7ff539600640 1 -- 172.21.15.32:0/3676408714 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c036070 con 0x7ff53c0b4490 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7ff539600640 1 -- 172.21.15.32:0/3676408714 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52c030d00 con 0x7ff53c0b4490 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.527+0000 7ff539600640 1 -- 172.21.15.32:0/3676408714 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c03a620 con 0x7ff53c0b4490 2024-04-12T19:50:31.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7fddd7200640 1 -- 172.21.15.32:0/3923585795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 msgr2=0x7fddd0054140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7fddd7200640 1 --2- 172.21.15.32:0/3923585795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fddd8074520 tx=0x7fddd80a75b0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7f2fa6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fa809aa40 0x7f2fa809aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7f2fa6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fa809aa40 0x7f2fa809aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51590/0 (socket says 172.21.15.32:51590) 2024-04-12T19:50:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7f2fa6a00640 1 -- 172.21.15.32:0/3855513512 learned_addr learned my addr 172.21.15.32:0/3855513512 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7f6c3f400640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28003b70 0x7f6c28003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7fc091000640 1 Processor -- start 2024-04-12T19:50:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7fc091000640 1 -- start start 2024-04-12T19:50:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 -- 172.21.15.32:0/3676408714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 msgr2=0x7ff53c16fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 --2- 172.21.15.32:0/3676408714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 0x7ff53c16fc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0b38e0 tx=0x7ff52c02ff50 comp rx=0 tx=0).stop 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 -- 172.21.15.32:0/3676408714 shutdown_connections 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 --2- 172.21.15.32:0/3676408714 >> v2:172.21.15.169:3300/0 conn(0x7ff53c06ff00 0x7ff53c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 --2- 172.21.15.32:0/3676408714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c0b4490 0x7ff53c16fc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 -- 172.21.15.32:0/3676408714 >> 172.21.15.32:0/3676408714 conn(0x7ff53c0b28b0 msgr2=0x7ff53c0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.529+0000 7ff541a00640 1 -- 172.21.15.32:0/3676408714 shutdown_connections 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3ea00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3fe00640 1 -- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c28004800 msgr2=0x7f6c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3fe00640 1 -- 172.21.15.32:0/3322566945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28003b70 msgr2=0x7f6c28003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c28003b70 0x7f6c28003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f6c3fe00640 1 -- 172.21.15.32:0/3322566945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28003520 con 0x7f6c28005210 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7ff541a00640 1 -- 172.21.15.32:0/3676408714 wait complete. 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7fddd7200640 1 -- 172.21.15.32:0/3923585795 shutdown_connections 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7fddd7200640 1 --2- 172.21.15.32:0/3923585795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd0054140 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7fddd7200640 1 --2- 172.21.15.32:0/3923585795 >> v2:172.21.15.169:3300/0 conn(0x7fddd003c0a0 0x7fddd00e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7fddd7200640 1 -- 172.21.15.32:0/3923585795 >> 172.21.15.32:0/3923585795 conn(0x7fddd0058ee0 msgr2=0x7fddd00592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f2fa6a00640 1 -- 172.21.15.32:0/3855513512 >> v2:172.21.15.32:3300/0 conn(0x7f2fa815c710 msgr2=0x7f2fa80b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f2fa6a00640 1 --2- 172.21.15.32:0/3855513512 >> v2:172.21.15.32:3300/0 conn(0x7f2fa815c710 0x7f2fa80b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f2fa6a00640 1 -- 172.21.15.32:0/3855513512 >> v1:172.21.15.32:6789/0 conn(0x7f2fa80b08c0 legacy=0x7f2fa80b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.530+0000 7f2fa6a00640 1 -- 172.21.15.32:0/3855513512 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa80afde0 con 0x7f2fa809aa40 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.531+0000 7fc091000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08403e840 0x7fc084053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7f2fa6a00640 1 --2- 172.21.15.32:0/3855513512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa809aa40 0x7f2fa809aea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fa806ffb0 tx=0x7f2f9802b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c463bbc7cdeeb81a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c2800b9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c4006a880 tx=0x7f6c28005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19399829c3b8eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.531+0000 7fc091000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc084054130 0x7fc0840068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.531+0000 7fc091000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc084058570 con 0x7fc0840e30c0 2024-04-12T19:50:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.531+0000 7fc091000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc084058850 con 0x7fc08403e840 2024-04-12T19:50:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.531+0000 7fc091000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc084058b30 con 0x7fc084054130 2024-04-12T19:50:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08403e840 0x7fc084053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08403e840 0x7fc084053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32772/0 (socket says 172.21.15.32:32772) 2024-04-12T19:50:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 -- 172.21.15.32:0/1091885076 learned_addr learned my addr 172.21.15.32:0/1091885076 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08be00640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc084054130 0x7fc0840068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fddd7200640 1 -- 172.21.15.32:0/3923585795 shutdown_connections 2024-04-12T19:50:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fddd7200640 1 -- 172.21.15.32:0/3923585795 wait complete. 2024-04-12T19:50:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b4070 con 0x7f6c28005210 2024-04-12T19:50:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.533+0000 7ff541a00640 1 Processor -- start 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.533+0000 7ff541a00640 1 -- start start 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 -- 172.21.15.32:0/1091885076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc084054130 msgr2=0x7fc0840068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc084054130 0x7fc0840068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 -- 172.21.15.32:0/1091885076 >> v1:172.21.15.169:6789/0 conn(0x7fc0840e30c0 legacy=0x7fc084057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.532+0000 7fc08aa00640 1 -- 172.21.15.32:0/1091885076 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0840063b0 con 0x7fc08403e840 2024-04-12T19:50:31.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.533+0000 7f2fa6000640 1 -- 172.21.15.32:0/3855513512 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98034070 con 0x7f2fa809aa40 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7f2fa6000640 1 -- 172.21.15.32:0/3855513512 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f98030870 con 0x7f2fa809aa40 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7f2fa6000640 1 -- 172.21.15.32:0/3855513512 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98030c40 con 0x7f2fa809aa40 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.533+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c400af440 con 0x7f6c28005210 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.533+0000 7f6c3e000640 1 -- 172.21.15.32:0/3322566945 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40050ca0 con 0x7f6c28005210 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 msgr2=0x7f6c2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7f6c44a00640 1 --2- 172.21.15.32:0/3322566945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c2800b9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c4006a880 tx=0x7f6c28005830 comp rx=0 tx=0).stop 2024-04-12T19:50:31.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7f6c3fe00640 1 -- 172.21.15.32:0/3322566945 reap_dead start 2024-04-12T19:50:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7fc08aa00640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08403e840 0x7fc084053bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc078009870 tx=0x7fc07802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93aab8130d8add server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7fc08a000640 1 -- 172.21.15.32:0/1091885076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07802f830 con 0x7fc08403e840 2024-04-12T19:50:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7fc08a000640 1 -- 172.21.15.32:0/1091885076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07802f9d0 con 0x7fc08403e840 2024-04-12T19:50:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.534+0000 7fc08a000640 1 -- 172.21.15.32:0/1091885076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07802fda0 con 0x7fc08403e840 2024-04-12T19:50:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 shutdown_connections 2024-04-12T19:50:31.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 -- 172.21.15.32:0/3855513512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa809aa40 msgr2=0x7f2fa809aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 --2- 172.21.15.32:0/3855513512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa809aa40 0x7f2fa809aea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fa806ffb0 tx=0x7f2f9802b9b0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 >> 172.21.15.32:0/3322566945 conn(0x7f6c38056ca0 msgr2=0x7f6c380570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7fc091000640 1 -- 172.21.15.32:0/1091885076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08403e840 msgr2=0x7fc084053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7fc091000640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08403e840 0x7fc084053bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc078009870 tx=0x7fc07802ba10 comp rx=0 tx=0).stop 2024-04-12T19:50:31.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 -- 172.21.15.32:0/3855513512 shutdown_connections 2024-04-12T19:50:31.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 --2- 172.21.15.32:0/3855513512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa809aa40 0x7f2fa809aea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 --2- 172.21.15.32:0/3855513512 >> v2:172.21.15.32:3300/0 conn(0x7f2fa815c710 0x7f2fa80b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7f2fad200640 1 -- 172.21.15.32:0/3855513512 >> 172.21.15.32:0/3855513512 conn(0x7f2fa80b40a0 msgr2=0x7f2fa80b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c06ff00 0x7ff53c163360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff53c0b4490 0x7ff53c164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff53c16b9c0 0x7ff53c16a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c16c560 con 0x7ff53c16b9c0 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c0b3f70 con 0x7ff53c0b4490 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.535+0000 7ff541a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff53c16c6e0 con 0x7ff53c06ff00 2024-04-12T19:50:31.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fddd7200640 1 Processor -- start 2024-04-12T19:50:31.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7f2fad200640 1 -- 172.21.15.32:0/3855513512 shutdown_connections 2024-04-12T19:50:31.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 -- 172.21.15.32:0/1091885076 shutdown_connections 2024-04-12T19:50:31.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 shutdown_connections 2024-04-12T19:50:31.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc084054130 0x7fc0840068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 --2- 172.21.15.32:0/1091885076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08403e840 0x7fc084053bf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7f6c44a00640 1 -- 172.21.15.32:0/3322566945 wait complete. 2024-04-12T19:50:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 -- 172.21.15.32:0/1091885076 >> 172.21.15.32:0/1091885076 conn(0x7fc0840559e0 msgr2=0x7fc084055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 -- 172.21.15.32:0/1091885076 shutdown_connections 2024-04-12T19:50:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7f2fad200640 1 -- 172.21.15.32:0/3855513512 wait complete. 2024-04-12T19:50:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fc091000640 1 -- 172.21.15.32:0/1091885076 wait complete. 2024-04-12T19:50:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7fddd7200640 1 -- start start 2024-04-12T19:50:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.537+0000 7fe5e3e00640 1 Processor -- start 2024-04-12T19:50:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.538+0000 7fe5e3e00640 1 -- start start 2024-04-12T19:50:31.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.538+0000 7fddd7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd003c0a0 0x7fddd00e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.536+0000 7ff53a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff53c0b4490 0x7ff53c164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd00df800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddd00e3160 0x7fddd00e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00e3540 con 0x7fddd003c0a0 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0006640 con 0x7fddd00e3160 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fddd0006b60 con 0x7fddd0053cb0 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd003c0a0 0x7fddd00e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.537+0000 7ff53b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff53c16b9c0 0x7ff53c16a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.537+0000 7ff53b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff53c16b9c0 0x7ff53c16a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32784/0 (socket says 172.21.15.32:32784) 2024-04-12T19:50:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.537+0000 7ff53b400640 1 -- 172.21.15.32:0/3903599543 learned_addr learned my addr 172.21.15.32:0/3903599543 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7ff53a000640 1 -- 172.21.15.32:0/3903599543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c06ff00 msgr2=0x7ff53c163360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7ff53a000640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff53c06ff00 0x7ff53c163360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5dc0b6ae0 0x7fe5dc169870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7ff53a000640 1 -- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff53c16b9c0 msgr2=0x7ff53c16a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7ff53a000640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff53c16b9c0 0x7ff53c16a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7ff53a000640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c163a00 con 0x7ff53c0b4490 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fe5e3e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc0b8f30 con 0x7fe5dc0b8a90 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fe5e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc0b9210 con 0x7fe5dc15ec60 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fe5e3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc0b94f0 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fe5e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5dc0b6ae0 0x7fe5dc169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fe5e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5dc0b6ae0 0x7fe5dc169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32804/0 (socket says 172.21.15.32:32804) 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fe5e1e00640 1 -- 172.21.15.32:0/412480803 learned_addr learned my addr 172.21.15.32:0/412480803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7ff53a000640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff53c0b4490 0x7ff53c164470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e850 tx=0x7ff52800ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fc091000640 1 Processor -- start 2024-04-12T19:50:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd003c0a0 0x7fddd00e40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32798/0 (socket says 172.21.15.32:32798) 2024-04-12T19:50:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd5e00640 1 -- 172.21.15.32:0/3224647138 learned_addr learned my addr 172.21.15.32:0/3224647138 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.539+0000 7fddd5400640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd00df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd6800640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddd00e3160 0x7fddd00e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fc091000640 1 -- start start 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd5400640 1 -- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddd00e3160 msgr2=0x7fddd00e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd5400640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddd00e3160 0x7fddd00e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd5400640 1 -- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd003c0a0 msgr2=0x7fddd00e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd5400640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddd003c0a0 0x7fddd00e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fddd5400640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd00e0270 con 0x7fddd0053cb0 2024-04-12T19:50:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fe5e1e00640 1 -- 172.21.15.32:0/412480803 >> v1:172.21.15.169:6789/0 conn(0x7fe5dc15ec60 legacy=0x7fe5dc0ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.540+0000 7fe5e1e00640 1 -- 172.21.15.32:0/412480803 >> v1:172.21.15.32:6789/0 conn(0x7fe5dc0b8a90 legacy=0x7fe5dc0b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7fe5e1e00640 1 -- 172.21.15.32:0/412480803 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc0b97d0 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7fddd6800640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fddd00e3160 0x7fddd00e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7f2fad200640 1 Processor -- start 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7f1258c00640 1 Processor -- start 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528024070 con 0x7ff53c0b4490 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53c16edc0 con 0x7ff53c0b4490 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53c163f50 con 0x7ff53c0b4490 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff528017c60 con 0x7ff53c0b4490 2024-04-12T19:50:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528024070 con 0x7ff53c0b4490 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.541+0000 7f1258c00640 1 -- start start 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7f6c44a00640 1 Processor -- start 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 0x7fc084052790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc084054130 0x7fc0840d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0840e30c0 0x7fc0840e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc084006e10 con 0x7fc084054130 2024-04-12T19:50:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0840e34a0 con 0x7fc08403e840 2024-04-12T19:50:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fc091000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc084006f90 con 0x7fc0840e30c0 2024-04-12T19:50:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7f2fad200640 1 -- start start 2024-04-12T19:50:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fe5e1e00640 1 --2- 172.21.15.32:0/412480803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b6ae0 0x7fe5dc169870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009e50 tx=0x7fe5d402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae01d7ac3de81b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.542+0000 7fddd5400640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd00df800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e850 tx=0x7fddc400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fc08b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 0x7fc084052790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fc08b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 0x7fc084052790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51638/0 (socket says 172.21.15.32:51638) 2024-04-12T19:50:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fc08b400640 1 -- 172.21.15.32:0/3633800875 learned_addr learned my addr 172.21.15.32:0/3633800875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7f6c44a00640 1 -- start start 2024-04-12T19:50:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fe5dbe00640 1 -- 172.21.15.32:0/412480803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d402cb10 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fe5dbe00640 1 -- 172.21.15.32:0/412480803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d402ccb0 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fe5dbe00640 1 -- 172.21.15.32:0/412480803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d402ce60 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fc08aa00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc084054130 0x7fc0840d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7f1258c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1254072810 0x7f12540b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7f1258c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540b01a0 con 0x7f1254162e90 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7f1258c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540b3900 con 0x7f12540afd70 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7f1258c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f12540b3be0 con 0x7f1254072810 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff514001030 con 0x7ff53c0b4490 2024-04-12T19:50:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fc08be00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0840e30c0 0x7fc0840e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4017070 con 0x7fddd0053cb0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc400c940 con 0x7fddd0053cb0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc400cd10 con 0x7fddd0053cb0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.543+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd00e4610 con 0x7fddd0053cb0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd00e3ba0 con 0x7fddd0053cb0 2024-04-12T19:50:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08b400640 1 -- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0840e30c0 msgr2=0x7fc0840e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7f1251400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1254072810 0x7f12540b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7f1251400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1254072810 0x7f12540b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32826/0 (socket says 172.21.15.32:32826) 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7f1251400640 1 -- 172.21.15.32:0/198384848 learned_addr learned my addr 172.21.15.32:0/198384848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fe5e3e00640 1 -- 172.21.15.32:0/412480803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b6ae0 msgr2=0x7fe5dc169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fe5e3e00640 1 --2- 172.21.15.32:0/412480803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b6ae0 0x7fe5dc169870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009e50 tx=0x7fe5d402bf70 comp rx=0 tx=0).stop 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7f1251400640 1 -- 172.21.15.32:0/198384848 >> v1:172.21.15.169:6789/0 conn(0x7f12540afd70 legacy=0x7f12540ce790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-12T19:50:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7f1251400640 1 -- 172.21.15.32:0/198384848 >> v1:172.21.15.32:6789/0 conn(0x7f1254162e90 legacy=0x7f1254072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fe5e3e00640 1 -- 172.21.15.32:0/412480803 shutdown_connections 2024-04-12T19:50:31.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fe5e3e00640 1 --2- 172.21.15.32:0/412480803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b6ae0 0x7fe5dc169870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fe5e3e00640 1 -- 172.21.15.32:0/412480803 >> 172.21.15.32:0/412480803 conn(0x7fe5dc073480 msgr2=0x7fe5dc073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7f2fad200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 0x7f2fa814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddd0005390 con 0x7fddd0053cb0 2024-04-12T19:50:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7f2fad200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 0x7f2fa814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7f6c44a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c381762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7f2fad200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa815c710 0x7f2fa8157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7ff38d200640 1 Processor -- start 2024-04-12T19:50:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7f2fa6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 0x7f2fa814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c44a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c38179f30 0x7f6c3817a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08b400640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0840e30c0 0x7fc0840e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08b400640 1 -- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc084054130 msgr2=0x7fc0840d9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08b400640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc084054130 0x7fc0840d9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08b400640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08404f120 con 0x7fc08403e840 2024-04-12T19:50:31.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff528027070 con 0x7ff53c0b4490 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.544+0000 7fc08be00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0840e30c0 0x7fc0840e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c44a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380e2a70 0x7f6c380e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c44a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c380549f0 con 0x7f6c380e2a70 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c44a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38054870 con 0x7f6c28005210 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c44a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c380516b0 con 0x7f6c38179f30 2024-04-12T19:50:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c3f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c381762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7ff532a00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff51c06caf0 0x7ff51c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff528093860 con 0x7ff53c0b4490 2024-04-12T19:50:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff532a00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7ff51c074c90 0x7ff51c0770b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff51c077760 con 0x7ff51c074c90 2024-04-12T19:50:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f1251400640 1 -- 172.21.15.32:0/198384848 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540b3ec0 con 0x7f1254072810 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.546+0000 7ff38d200640 1 -- start start 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff38d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3880cf9b0 0x7ff388163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff38d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880d48a0 0x7ff3880d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff38d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3880d2d20 con 0x7ff388161fc0 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff38d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff3880d5220 con 0x7ff3880cf9b0 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff38d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff3880cff70 con 0x7ff3880d48a0 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c3f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c381762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51660/0 (socket says 172.21.15.32:51660) 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f6c3f400640 1 -- 172.21.15.32:0/1459502259 learned_addr learned my addr 172.21.15.32:0/1459502259 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380e2a70 0x7f6c380e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7fe5e3e00640 1 -- 172.21.15.32:0/412480803 shutdown_connections 2024-04-12T19:50:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3ea00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c38179f30 0x7f6c3817a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff52805d8e0 con 0x7ff53c0b4490 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff53aa00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff51c06caf0 0x7ff51c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff385400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3880cf9b0 0x7ff388163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff385400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3880cf9b0 0x7ff388163090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51670/0 (socket says 172.21.15.32:51670) 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7ff385400640 1 -- 172.21.15.32:0/805288883 learned_addr learned my addr 172.21.15.32:0/805288883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3f400640 1 -- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c38179f30 msgr2=0x7f6c3817a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3f400640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c38179f30 0x7f6c3817a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7fe5e3e00640 1 -- 172.21.15.32:0/412480803 wait complete. 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f2fa6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 0x7f2fa814ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32842/0 (socket says 172.21.15.32:32842) 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f2fa7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 0x7f2fa814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f2fa7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 0x7f2fa814cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:54964/0 (socket says 172.21.15.32:54964) 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f2fa7400640 1 -- 172.21.15.32:0/419292190 learned_addr learned my addr 172.21.15.32:0/419292190 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f2fa7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa815c710 0x7f2fa8157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.547+0000 7f2fad200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80b0430 con 0x7f2fa80b08c0 2024-04-12T19:50:31.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3f400640 1 -- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380e2a70 msgr2=0x7f6c380e2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3f400640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380e2a70 0x7f6c380e2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7f6c3f400640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c38176810 con 0x7f6c28005210 2024-04-12T19:50:31.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.548+0000 7fc08b400640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 0x7fc084052790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c430 tx=0x7fc08000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa80b6bc0 con 0x7f2fa815c710 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa815caf0 con 0x7f2fa809aa40 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080017070 con 0x7fc08403e840 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0800052d0 con 0x7fc08403e840 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080003830 con 0x7fc08403e840 2024-04-12T19:50:31.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7ff53b400640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7ff51c074c90 0x7ff51c0770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.549+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddc401a070 con 0x7fddd0053cb0 2024-04-12T19:50:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fddce000640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddb806c940 0x7fddb806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fddc408bcc0 con 0x7fddd0053cb0 2024-04-12T19:50:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fddce000640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7fddb8074b60 0x7fddb8076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f6c3f400640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c381762d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001790 tx=0x7f6c2c001b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddb8077630 con 0x7fddb8074b60 2024-04-12T19:50:31.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fddc408c1a0 con 0x7fddd0053cb0 2024-04-12T19:50:31.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7fddd6800640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7fddb8074b60 0x7fddb8076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa6a00640 1 -- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 msgr2=0x7f2fa814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa6a00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 0x7f2fa814cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa6a00640 1 -- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa815c710 msgr2=0x7f2fa8157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa6a00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa815c710 0x7f2fa8157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa6a00640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa814f510 con 0x7f2fa80b08c0 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7f2fa7400640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fa809aa40 0x7f2fa814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f2fa7e00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa815c710 0x7f2fa8157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00f7b0 con 0x7f6c28005210 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c018070 con 0x7f6c28005210 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00fb70 con 0x7f6c28005210 2024-04-12T19:50:31.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f6c44a00640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c380e3760 con 0x7f6c28005210 2024-04-12T19:50:31.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08403c540 con 0x7fc08403e840 2024-04-12T19:50:31.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08403cb00 con 0x7fc08403e840 2024-04-12T19:50:31.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7ff53b400640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7ff51c074c90 0x7ff51c0770b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.550+0000 7fe5e3e00640 1 Processor -- start 2024-04-12T19:50:31.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7fe5e3e00640 1 -- start start 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f1251400640 1 --2- 172.21.15.32:0/198384848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254072810 0x7f12540b58e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1240009870 tx=0x7f124002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1ffa927c78197c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7ff385400640 1 -- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880d48a0 msgr2=0x7ff3880d4ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7ff385400640 1 --2- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880d48a0 0x7ff3880d4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7ff385400640 1 -- 172.21.15.32:0/805288883 >> v1:172.21.15.169:6789/0 conn(0x7ff388161fc0 legacy=0x7ff3880b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7ff385400640 1 -- 172.21.15.32:0/805288883 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3880d2fc0 con 0x7ff3880cf9b0 2024-04-12T19:50:31.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.551+0000 7f2fa6a00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 0x7f2fa814ebc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f98033000 tx=0x7f2f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7fddd5e00640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddb806c940 0x7fddb806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f1250a00640 1 -- 172.21.15.32:0/198384848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240035020 con 0x7f1254072810 2024-04-12T19:50:31.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f1250a00640 1 -- 172.21.15.32:0/198384848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240031870 con 0x7f1254072810 2024-04-12T19:50:31.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f1250a00640 1 -- 172.21.15.32:0/198384848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240031c40 con 0x7f1254072810 2024-04-12T19:50:31.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc0840057e0 con 0x7fc08403e840 2024-04-12T19:50:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98034040 con 0x7f2fa80b08c0 2024-04-12T19:50:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f980433f0 con 0x7f2fa80b08c0 2024-04-12T19:50:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980437e0 con 0x7f2fa80b08c0 2024-04-12T19:50:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa815ce50 con 0x7f2fa80b08c0 2024-04-12T19:50:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.552+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa815d410 con 0x7f2fa80b08c0 2024-04-12T19:50:31.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 0x7fe5dc15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b8a90 0x7fe5dc1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 -- 172.21.15.32:0/198384848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254072810 msgr2=0x7f12540b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 --2- 172.21.15.32:0/198384848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254072810 0x7f12540b58e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1240009870 tx=0x7f124002b910 comp rx=0 tx=0).stop 2024-04-12T19:50:31.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 -- 172.21.15.32:0/198384848 shutdown_connections 2024-04-12T19:50:31.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5dc15ec60 0x7fe5dc0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 --2- 172.21.15.32:0/198384848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254072810 0x7f12540b58e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 -- 172.21.15.32:0/198384848 >> 172.21.15.32:0/198384848 conn(0x7f12540b2460 msgr2=0x7f12540b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d3c90 con 0x7fe5dc0b8a90 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 0x7fe5dc15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc15f040 con 0x7fe5dc15ec60 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 0x7fe5dc15a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:54978/0 (socket says 172.21.15.32:54978) 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e1400640 1 -- 172.21.15.32:0/947302029 learned_addr learned my addr 172.21.15.32:0/947302029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d3e10 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 -- 172.21.15.32:0/198384848 shutdown_connections 2024-04-12T19:50:31.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f1258c00640 1 -- 172.21.15.32:0/198384848 wait complete. 2024-04-12T19:50:31.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.553+0000 7f6c44a00640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c380e3d20 con 0x7f6c28005210 2024-04-12T19:50:31.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.554+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fa80755c0 con 0x7f2fa80b08c0 2024-04-12T19:50:31.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c3817a8b0 con 0x7f6c28005210 2024-04-12T19:50:31.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== osd.2 v2:172.21.15.32:6820/4204074222 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff51c077760 con 0x7ff51c074c90 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7ff53aa00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff51c06caf0 0x7ff51c06efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff52c03b000 tx=0x7ff52c036000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fddd5e00640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddb806c940 0x7fddb806ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fddc8014f70 tx=0x7fddc8016040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.554+0000 7fe5e0a00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b8a90 0x7fe5dc1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.554+0000 7fe5e1e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5dc15ec60 0x7fe5dc0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e1400640 1 -- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5dc15ec60 msgr2=0x7fe5dc0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e1400640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5dc15ec60 0x7fe5dc0c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e1400640 1 -- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b8a90 msgr2=0x7fe5dc1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e1400640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b8a90 0x7fe5dc1f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e1400640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc0ce860 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7fe5e0a00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5dc0b8a90 0x7fe5dc1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.555+0000 7f1258c00640 1 Processor -- start 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7f1258c00640 1 -- start start 2024-04-12T19:50:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7ff385400640 1 --2- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880cf9b0 0x7ff388163090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3880baa90 tx=0x7ff37402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a3d54fa18412d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc08001a070 con 0x7fc08403e840 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7fddd6800640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7fddb8074b60 0x7fddb8076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddb8077630 con 0x7fddb8074b60 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7fe5e1400640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 0x7fe5dc15a560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c400 tx=0x7fe5d000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d00052d0 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0003830 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1ec630 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1ecbf0 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.556+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f9804a060 con 0x7f2fa80b08c0 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1254072810 0x7f12541f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12540afd70 0x7f1254203f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254162e90 0x7f12540b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5dc068c80 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f12540cf020 con 0x7f1254162e90 2024-04-12T19:50:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12540ceea0 con 0x7f1254072810 2024-04-12T19:50:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1258c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12540cf1a0 con 0x7f12540afd70 2024-04-12T19:50:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7ff384a00640 1 -- 172.21.15.32:0/805288883 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374035020 con 0x7ff3880cf9b0 2024-04-12T19:50:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1252800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254162e90 0x7f12540b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.557+0000 7ff384a00640 1 -- 172.21.15.32:0/805288883 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff374031870 con 0x7ff3880cf9b0 2024-04-12T19:50:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1252800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254162e90 0x7f12540b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32866/0 (socket says 172.21.15.32:32866) 2024-04-12T19:50:31.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1252800640 1 -- 172.21.15.32:0/747020731 learned_addr learned my addr 172.21.15.32:0/747020731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7f1251e00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1254072810 0x7f12541f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12540afd70 0x7f1254203f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.558+0000 7ff384a00640 1 -- 172.21.15.32:0/805288883 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374031c40 con 0x7ff3880cf9b0 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 -- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1254072810 msgr2=0x7f12541f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1254072810 0x7f12541f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 -- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254162e90 msgr2=0x7f12540b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1254162e90 0x7f12540b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.559+0000 7f1251400640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1254201580 con 0x7f12540afd70 2024-04-12T19:50:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 -- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880cf9b0 msgr2=0x7ff388163090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 --2- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880cf9b0 0x7ff388163090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3880baa90 tx=0x7ff37402b910 comp rx=0 tx=0).stop 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 -- 172.21.15.32:0/805288883 shutdown_connections 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 --2- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880d48a0 0x7ff3880d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 --2- 172.21.15.32:0/805288883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880cf9b0 0x7ff388163090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 -- 172.21.15.32:0/805288883 >> 172.21.15.32:0/805288883 conn(0x7ff388073480 msgr2=0x7ff388073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7fc07f400640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc06c06c940 0x7fc06c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 -- 172.21.15.32:0/805288883 shutdown_connections 2024-04-12T19:50:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7f1251400640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12540afd70 0x7f1254203f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f12400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12400443f0 con 0x7f12540afd70 2024-04-12T19:50:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12400445b0 con 0x7f12540afd70 2024-04-12T19:50:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12400449a0 con 0x7f12540afd70 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc08008b4c0 con 0x7fc08403e840 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7fc07f400640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fc06c074bc0 0x7fc06c076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc06c077690 con 0x7fc06c074bc0 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc08008b9a0 con 0x7fc08403e840 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7fc08aa00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc06c06c940 0x7fc06c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.561+0000 7fc08be00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fc06c074bc0 0x7fc06c076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c2c03a690 con 0x7f6c28005210 2024-04-12T19:50:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.560+0000 7ff38d200640 1 -- 172.21.15.32:0/805288883 wait complete. 2024-04-12T19:50:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1254201860 con 0x7f12540afd70 2024-04-12T19:50:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1254202100 con 0x7f12540afd70 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f2f9f400640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f8406c920 0x7f2f8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f980ae380 con 0x7f2fa80b08c0 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f2f9f400640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f2f84074ac0 0x7f2f84076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f84077590 con 0x7f2f84074ac0 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2f980ae730 con 0x7f2fa80b08c0 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.562+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5d001a070 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7ff38d200640 1 Processor -- start 2024-04-12T19:50:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1254201c60 con 0x7f12540afd70 2024-04-12T19:50:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7f2fa7400640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f8406c920 0x7f2f8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7ff38d200640 1 -- start start 2024-04-12T19:50:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7f2fa7e00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f2f84074ac0 0x7f2f84076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7fe5d9600640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe5bc06c920 0x7fe5bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.563+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe5d008b510 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7fc08aa00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc06c06c940 0x7fc06c06ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc078036000 tx=0x7fc078008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7fe5d9600640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fe5bc074b40 0x7fe5bc076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5bc077610 con 0x7fe5bc074b40 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe5d008b900 con 0x7fe5dc0b6ae0 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7fe5e0a00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe5bc06c920 0x7fe5bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7fc08be00640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fc06c074bc0 0x7fc06c076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7fe5e1e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fe5bc074b40 0x7fe5bc076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff38d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880cf9b0 0x7ff388159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff38d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 0x7ff388148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff38d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 0x7ff388148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f124004b060 con 0x7f12540afd70 2024-04-12T19:50:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff386800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 0x7ff388148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff386800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 0x7ff388148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32878/0 (socket says 172.21.15.32:32878) 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7ff385400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 0x7ff388148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7ff385400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 0x7ff388148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51692/0 (socket says 172.21.15.32:51692) 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7ff386800640 1 -- 172.21.15.32:0/2087528192 learned_addr learned my addr 172.21.15.32:0/2087528192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881623a0 con 0x7ff388161fc0 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880d3580 con 0x7ff3880d48a0 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff3880d3aa0 con 0x7ff3880cf9b0 2024-04-12T19:50:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.564+0000 7f6c33400640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6c2006c8f0 0x7f6c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c2c08be70 con 0x7f6c28005210 2024-04-12T19:50:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7f6c33400640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7f6c20074a90 0x7f6c20076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c2c01ee50 con 0x7f6c20074a90 2024-04-12T19:50:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.565+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c08c260 con 0x7f6c28005210 2024-04-12T19:50:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7fe5e1e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fe5bc074b40 0x7fe5bc076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7f124a000640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f123406c8f0 0x7f123406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.566+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f12400ae310 con 0x7f12540afd70 2024-04-12T19:50:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f124a000640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1234074a90 0x7f1234076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== osd.1 v2:172.21.15.32:6800/4074798626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc06c077690 con 0x7fc06c074bc0 2024-04-12T19:50:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1234077560 con 0x7f1234074a90 2024-04-12T19:50:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f12400ae6c0 con 0x7f12540afd70 2024-04-12T19:50:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f1251e00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f123406c8f0 0x7f123406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff386800640 1 -- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880cf9b0 msgr2=0x7ff388159b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff386800640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3880cf9b0 0x7ff388159b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff386800640 1 -- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 msgr2=0x7ff388148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff386800640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 0x7ff388148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff386800640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38815bae0 con 0x7ff388161fc0 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7f6c20074a90 0x7f6c20076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7f2fa7400640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f8406c920 0x7f2f8406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fa814d370 tx=0x7f2f900091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7ff385400640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3880d48a0 0x7ff388148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.567+0000 7fe5e0a00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe5bc06c920 0x7fe5bc06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0cb4c0 tx=0x7fe5c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7f2fa7e00640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f2f84074ac0 0x7f2f84076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7ff386800640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 0x7ff388148cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff38000b9c0 tx=0x7ff38000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7f6c3ea00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6c2006c8f0 0x7f6c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7f6c3fe00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7f6c20074a90 0x7f6c20076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380019070 con 0x7ff388161fc0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38000cba0 con 0x7ff388161fc0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38815bdc0 con 0x7ff388161fc0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.568+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38815c380 con 0x7ff388161fc0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.569+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380014440 con 0x7ff388161fc0 2024-04-12T19:50:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.569+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff360001030 con 0x7ff388161fc0 2024-04-12T19:50:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.570+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== osd.5 v2:172.21.15.169:6816/2535062120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5bc077610 con 0x7fe5bc074b40 2024-04-12T19:50:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.570+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== osd.0 v2:172.21.15.32:6816/737701280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c2c01ee50 con 0x7f6c20074a90 2024-04-12T19:50:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.570+0000 7f6c3ea00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6c2006c8f0 0x7f6c2006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c340017e0 tx=0x7f6c34001470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.571+0000 7f1252800640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1234074a90 0x7f1234076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.570+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== osd.4 v2:172.21.15.169:6800/3921333402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f84077590 con 0x7f2f84074ac0 2024-04-12T19:50:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.571+0000 7f1251e00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f123406c8f0 0x7f123406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1244000f60 tx=0x7f1244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.572+0000 7f1252800640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1234074a90 0x7f1234076eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== osd.6 v2:172.21.15.169:6808/2726062170 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1234077560 con 0x7f1234074a90 2024-04-12T19:50:31.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.573+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff38001c070 con 0x7ff388161fc0 2024-04-12T19:50:31.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7ff37e000640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff36806c8f0 0x7ff36806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff38008bf90 con 0x7ff388161fc0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7ff37e000640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7ff368074b70 0x7ff368076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff380022ec0 con 0x7ff368074b70 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.574+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff38008c380 con 0x7ff388161fc0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.575+0000 7ff385e00640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff36806c8f0 0x7ff36806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.576+0000 7ff385400640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7ff368074b70 0x7ff368076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.577+0000 7ff385e00640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff36806c8f0 0x7ff36806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff378009620 tx=0x7ff378006b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.578+0000 7ff385400640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7ff368074b70 0x7ff368076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.581+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== osd.7 v2:172.21.15.169:6822/883923800 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff380022ec0 con 0x7ff368074b70 2024-04-12T19:50:31.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.588+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 --> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff514002dd0 con 0x7ff51c074c90 2024-04-12T19:50:31.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.589+0000 7ff532a00640 1 -- 172.21.15.32:0/3903599543 <== osd.2 v2:172.21.15.32:6820/4204074222 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff514002dd0 con 0x7ff51c074c90 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7ff51c074c90 msgr2=0x7ff51c0770b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7ff51c074c90 0x7ff51c0770b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff51c06caf0 msgr2=0x7ff51c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff51c06caf0 0x7ff51c06efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff52c03b000 tx=0x7ff52c036000 comp rx=0 tx=0).stop 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff53c0b4490 msgr2=0x7ff53c164470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.591+0000 7ff541a00640 1 --2- 172.21.15.32:0/3903599543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff53c0b4490 0x7ff53c164470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e850 tx=0x7ff52800ed10 comp rx=0 tx=0).stop 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.592+0000 7ff53b400640 1 -- 172.21.15.32:0/3903599543 reap_dead start 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.592+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 shutdown_connections 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.592+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 >> 172.21.15.32:0/3903599543 conn(0x7ff53c0b28b0 msgr2=0x7ff53c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.592+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 shutdown_connections 2024-04-12T19:50:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.592+0000 7ff541a00640 1 -- 172.21.15.32:0/3903599543 wait complete. 2024-04-12T19:50:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.595+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fddd00e3940 con 0x7fddb8074b60 2024-04-12T19:50:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.596+0000 7fddce000640 1 -- 172.21.15.32:0/3224647138 <== osd.3 v2:172.21.15.32:6808/3914231922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fddd80b4040 con 0x7fddb8074b60 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.596+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7fddb8074b60 msgr2=0x7fddb8076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.596+0000 7fddd7200640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7fddb8074b60 0x7fddb8076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.597+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddb806c940 msgr2=0x7fddb806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.597+0000 7fddd7200640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddb806c940 0x7fddb806ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fddc8014f70 tx=0x7fddc8016040 comp rx=0 tx=0).stop 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.597+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 msgr2=0x7fddd00df800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.597+0000 7fddd7200640 1 --2- 172.21.15.32:0/3224647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fddd0053cb0 0x7fddd00df800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e850 tx=0x7fddc400ed10 comp rx=0 tx=0).stop 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.597+0000 7fddd6800640 1 -- 172.21.15.32:0/3224647138 reap_dead start 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.600+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 shutdown_connections 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.600+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 >> 172.21.15.32:0/3224647138 conn(0x7fddd0058ee0 msgr2=0x7fddd00398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.600+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 shutdown_connections 2024-04-12T19:50:31.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.600+0000 7fddd7200640 1 -- 172.21.15.32:0/3224647138 wait complete. 2024-04-12T19:50:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.600+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 --> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5dc0b5de0 con 0x7fe5bc074b40 2024-04-12T19:50:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.601+0000 7fe5d9600640 1 -- 172.21.15.32:0/947302029 <== osd.5 v2:172.21.15.169:6816/2535062120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe5dc0b5de0 con 0x7fe5bc074b40 2024-04-12T19:50:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.603+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 --> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc084053300 con 0x7fc06c074bc0 2024-04-12T19:50:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.602+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 --> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c380057b0 con 0x7f6c20074a90 2024-04-12T19:50:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.603+0000 7f6c33400640 1 -- 172.21.15.32:0/1459502259 <== osd.0 v2:172.21.15.32:6816/737701280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c380057b0 con 0x7f6c20074a90 2024-04-12T19:50:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.604+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7f6c20074a90 msgr2=0x7f6c20076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.604+0000 7f6c30c00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7f6c20074a90 0x7f6c20076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6c2006c8f0 msgr2=0x7f6c2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f6c30c00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6c2006c8f0 0x7f6c2006edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6c340017e0 tx=0x7f6c34001470 comp rx=0 tx=0).stop 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 msgr2=0x7f6c381762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f6c30c00640 1 --2- 172.21.15.32:0/1459502259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c28005210 0x7f6c381762d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001790 tx=0x7f6c2c001b90 comp rx=0 tx=0).stop 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.604+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fe5bc074b40 msgr2=0x7fe5bc076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.604+0000 7fe5e3e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fe5bc074b40 0x7fe5bc076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe5bc06c920 msgr2=0x7fe5bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe5bc06c920 0x7fe5bc06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0cb4c0 tx=0x7fe5c8009040 comp rx=0 tx=0).stop 2024-04-12T19:50:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 msgr2=0x7fe5dc15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 --2- 172.21.15.32:0/947302029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5dc0b6ae0 0x7fe5dc15a560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c400 tx=0x7fe5d000c8c0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.603+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 --> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1254068c80 con 0x7f1234074a90 2024-04-12T19:50:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.604+0000 7f124a000640 1 -- 172.21.15.32:0/747020731 <== osd.6 v2:172.21.15.169:6808/2726062170 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1254068c80 con 0x7f1234074a90 2024-04-12T19:50:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fc07f400640 1 -- 172.21.15.32:0/3633800875 <== osd.1 v2:172.21.15.32:6800/4074798626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc084053300 con 0x7fc06c074bc0 2024-04-12T19:50:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e1e00640 1 -- 172.21.15.32:0/947302029 reap_dead start 2024-04-12T19:50:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1234074a90 msgr2=0x7f1234076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1234074a90 0x7f1234076eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 shutdown_connections 2024-04-12T19:50:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 >> 172.21.15.32:0/947302029 conn(0x7fe5dc073480 msgr2=0x7fe5dc0b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f123406c8f0 msgr2=0x7f123406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f123406c8f0 0x7f123406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1244000f60 tx=0x7f1244009040 comp rx=0 tx=0).stop 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12540afd70 msgr2=0x7f1254203f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1258c00640 1 --2- 172.21.15.32:0/747020731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12540afd70 0x7f1254203f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1240033000 tx=0x7f12400046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 shutdown_connections 2024-04-12T19:50:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7fe5e3e00640 1 -- 172.21.15.32:0/947302029 wait complete. 2024-04-12T19:50:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.605+0000 7f1252800640 1 -- 172.21.15.32:0/747020731 reap_dead start 2024-04-12T19:50:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.606+0000 7f6c3fe00640 1 -- 172.21.15.32:0/1459502259 reap_dead start 2024-04-12T19:50:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.606+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 shutdown_connections 2024-04-12T19:50:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.606+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 >> 172.21.15.32:0/1459502259 conn(0x7f6c38056ca0 msgr2=0x7f6c3803a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.606+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 shutdown_connections 2024-04-12T19:50:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.606+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 >> 172.21.15.32:0/747020731 conn(0x7f12540b2460 msgr2=0x7f12540b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.607+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fc06c074bc0 msgr2=0x7fc06c076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.607+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 shutdown_connections 2024-04-12T19:50:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.607+0000 7fc091000640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fc06c074bc0 0x7fc06c076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.607+0000 7f1258c00640 1 -- 172.21.15.32:0/747020731 wait complete. 2024-04-12T19:50:31.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.608+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 shutdown_connections 2024-04-12T19:50:31.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.608+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 --> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fa80d2310 con 0x7f2f84074ac0 2024-04-12T19:50:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.608+0000 7f6c30c00640 1 -- 172.21.15.32:0/1459502259 wait complete. 2024-04-12T19:50:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff360002dd0 con 0x7ff368074b70 2024-04-12T19:50:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc06c06c940 msgr2=0x7fc06c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7fc091000640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc06c06c940 0x7fc06c06ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc078036000 tx=0x7fc078008030 comp rx=0 tx=0).stop 2024-04-12T19:50:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 msgr2=0x7fc084052790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7fc091000640 1 --2- 172.21.15.32:0/3633800875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc08403e840 0x7fc084052790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c430 tx=0x7fc08000c8f0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.609+0000 7f2f9f400640 1 -- 172.21.15.32:0/419292190 <== osd.4 v2:172.21.15.169:6800/3921333402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2fa80d2310 con 0x7f2f84074ac0 2024-04-12T19:50:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7fc08be00640 1 -- 172.21.15.32:0/3633800875 reap_dead start 2024-04-12T19:50:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 shutdown_connections 2024-04-12T19:50:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7ff37e000640 1 -- 172.21.15.32:0/2087528192 <== osd.7 v2:172.21.15.169:6822/883923800 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff360002dd0 con 0x7ff368074b70 2024-04-12T19:50:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 >> 172.21.15.32:0/3633800875 conn(0x7fc0840559e0 msgr2=0x7fc08403a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 shutdown_connections 2024-04-12T19:50:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.610+0000 7fc091000640 1 -- 172.21.15.32:0/3633800875 wait complete. 2024-04-12T19:50:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.611+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7ff368074b70 msgr2=0x7ff368076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.611+0000 7ff38d200640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7ff368074b70 0x7ff368076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.611+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f2f84074ac0 msgr2=0x7f2f84076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.611+0000 7f2fad200640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f2f84074ac0 0x7f2f84076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff36806c8f0 msgr2=0x7ff36806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff36806c8f0 0x7ff36806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff378009620 tx=0x7ff378006b30 comp rx=0 tx=0).stop 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 msgr2=0x7ff388148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 --2- 172.21.15.32:0/2087528192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff388161fc0 0x7ff388148cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff38000b9c0 tx=0x7ff38000be80 comp rx=0 tx=0).stop 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff386800640 1 -- 172.21.15.32:0/2087528192 reap_dead start 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 shutdown_connections 2024-04-12T19:50:31.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 >> 172.21.15.32:0/2087528192 conn(0x7ff388073480 msgr2=0x7ff3881e84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f8406c920 msgr2=0x7f2f8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7f2fad200640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f8406c920 0x7f2f8406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fa814d370 tx=0x7f2f900091c0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 msgr2=0x7f2fa814ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.612+0000 7f2fad200640 1 --2- 172.21.15.32:0/419292190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80b08c0 0x7f2fa814ebc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f98033000 tx=0x7f2f980046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.613+0000 7f2fa7e00640 1 -- 172.21.15.32:0/419292190 reap_dead start 2024-04-12T19:50:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.614+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 shutdown_connections 2024-04-12T19:50:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.614+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 shutdown_connections 2024-04-12T19:50:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.614+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 >> 172.21.15.32:0/419292190 conn(0x7f2fa80b40a0 msgr2=0x7f2fa80b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.614+0000 7ff38d200640 1 -- 172.21.15.32:0/2087528192 wait complete. 2024-04-12T19:50:31.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.615+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 shutdown_connections 2024-04-12T19:50:31.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.615+0000 7f2fad200640 1 -- 172.21.15.32:0/419292190 wait complete. 2024-04-12T19:50:31.641 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:31.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-12T19:50:31.643 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-12T19:50:31.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-12T19:50:31.644 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:31.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-12T19:50:31.645 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:31.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-12T19:50:31.646 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:31.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-12T19:50:31.647 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-12T19:50:31.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-12T19:50:31.648 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-12T19:50:31.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-12T19:50:31.649 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:31.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-12T19:50:31.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.979+0000 7fc925200640 1 Processor -- start 2024-04-12T19:50:31.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.980+0000 7fc925200640 1 -- start start 2024-04-12T19:50:31.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.981+0000 7fc925200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc9180df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.982+0000 7fc925200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc918054690 0x7fc9180dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.982+0000 7fc91e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc918054690 0x7fc9180dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc91e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc918054690 0x7fc9180dfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32882/0 (socket says 172.21.15.32:32882) 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc91e800640 1 -- 172.21.15.32:0/320881041 learned_addr learned my addr 172.21.15.32:0/320881041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc91d400640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc9180df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc918053310 con 0x7fc9180df130 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc9180535f0 con 0x7fc918054690 2024-04-12T19:50:31.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.983+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc9180538d0 con 0x7fc91800aad0 2024-04-12T19:50:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.984+0000 7fc91e800640 1 -- 172.21.15.32:0/320881041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 msgr2=0x7fc9180df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.984+0000 7fc91e800640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc9180df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.984+0000 7fc91e800640 1 -- 172.21.15.32:0/320881041 >> v1:172.21.15.169:6789/0 conn(0x7fc9180df130 legacy=0x7fc91804fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.984+0000 7fc91e800640 1 -- 172.21.15.32:0/320881041 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc918053bb0 con 0x7fc918054690 2024-04-12T19:50:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.986+0000 7fc91e800640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc9180dfa50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf5a70b89810068 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.986+0000 7fa15f800640 1 Processor -- start 2024-04-12T19:50:31.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.986+0000 7fa15f800640 1 -- start start 2024-04-12T19:50:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.987+0000 7fc91ca00640 1 -- 172.21.15.32:0/320881041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c02f830 con 0x7fc918054690 2024-04-12T19:50:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.987+0000 7fc91ca00640 1 -- 172.21.15.32:0/320881041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90c02f9d0 con 0x7fc918054690 2024-04-12T19:50:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.987+0000 7fc91ca00640 1 -- 172.21.15.32:0/320881041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c02fda0 con 0x7fc918054690 2024-04-12T19:50:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 msgr2=0x7fc9180dfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc9180dfa50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02ba10 comp rx=0 tx=0).stop 2024-04-12T19:50:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 shutdown_connections 2024-04-12T19:50:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc9180dfa50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 --2- 172.21.15.32:0/320881041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc9180df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 >> 172.21.15.32:0/320881041 conn(0x7fc918009fe0 msgr2=0x7fc91800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fa15f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1580b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.988+0000 7fa15f800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1580b4970 0x7fa1580ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15ce00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa158162df0 0x7fa158071f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43266/0 (socket says 172.21.15.32:43266) 2024-04-12T19:50:31.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15f800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1581631d0 con 0x7fa158162df0 2024-04-12T19:50:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15f800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1580b45e0 con 0x7fa1580b4970 2024-04-12T19:50:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa15806f770 con 0x7fa158072640 2024-04-12T19:50:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15ce00640 1 -- 172.21.15.32:0/1541051172 learned_addr learned my addr 172.21.15.32:0/1541051172 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.989+0000 7fa15d800640 1 --2- 172.21.15.32:0/1541051172 >> v2:172.21.15.32:3300/0 conn(0x7fa1580b4970 0x7fa1580ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 shutdown_connections 2024-04-12T19:50:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fc925200640 1 -- 172.21.15.32:0/320881041 wait complete. 2024-04-12T19:50:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fa15d800640 1 -- 172.21.15.32:0/1541051172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 msgr2=0x7fa1580b40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-12T19:50:31.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fa15d800640 1 --2- 172.21.15.32:0/1541051172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1580b40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fa15d800640 1 -- 172.21.15.32:0/1541051172 >> v1:172.21.15.32:6789/0 conn(0x7fa158162df0 legacy=0x7fa158071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.990+0000 7fa15d800640 1 -- 172.21.15.32:0/1541051172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15806f080 con 0x7fa1580b4970 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.992+0000 7fa15d800640 1 --2- 172.21.15.32:0/1541051172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ce450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1480097a0 tx=0x7fa14802b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468ecde778a99fe server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.993+0000 7fc925200640 1 Processor -- start 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.993+0000 7fc925200640 1 -- start start 2024-04-12T19:50:31.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.993+0000 7fa157400640 1 -- 172.21.15.32:0/1541051172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148037070 con 0x7fa1580b4970 2024-04-12T19:50:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.993+0000 7fa157400640 1 -- 172.21.15.32:0/1541051172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa148030710 con 0x7fa1580b4970 2024-04-12T19:50:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.993+0000 7fa157400640 1 -- 172.21.15.32:0/1541051172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148030ae0 con 0x7fa1580b4970 2024-04-12T19:50:31.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fa15f800640 1 -- 172.21.15.32:0/1541051172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 msgr2=0x7fa1580ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc918185ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 0x7fc91804ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9180e04e0 con 0x7fc918054690 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc918054170 con 0x7fc9180df130 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fc925200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc9180dff90 con 0x7fc91800aad0 2024-04-12T19:50:31.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.994+0000 7fa15f800640 1 --2- 172.21.15.32:0/1541051172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ce450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1480097a0 tx=0x7fa14802b730 comp rx=0 tx=0).stop 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 0x7fc91804ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc918185ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fa15f800640 1 -- 172.21.15.32:0/1541051172 shutdown_connections 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fa15f800640 1 --2- 172.21.15.32:0/1541051172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ce450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fa15f800640 1 --2- 172.21.15.32:0/1541051172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1580b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fa15f800640 1 -- 172.21.15.32:0/1541051172 >> 172.21.15.32:0/1541051172 conn(0x7fa15806b930 msgr2=0x7fa15806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32896/0 (socket says 172.21.15.32:32896) 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc918185ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:55010/0 (socket says 172.21.15.32:55010) 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.995+0000 7fc91de00640 1 -- 172.21.15.32:0/1798411770 learned_addr learned my addr 172.21.15.32:0/1798411770 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.996+0000 7fc91d400640 1 -- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 msgr2=0x7fc918185ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.996+0000 7fc91d400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc918185ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.996+0000 7fa15f800640 1 -- 172.21.15.32:0/1541051172 shutdown_connections 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.996+0000 7fc91d400640 1 -- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 msgr2=0x7fc91804ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.997+0000 7fc91d400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 0x7fc91804ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.997+0000 7fc91d400640 1 -- 172.21.15.32:0/1798411770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91804d340 con 0x7fc918054690 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.997+0000 7fc91e800640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 0x7fc91804ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.997+0000 7fc91d400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc91000e850 tx=0x7fc91000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.998+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910017070 con 0x7fc918054690 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91000c940 con 0x7fc918054690 2024-04-12T19:50:31.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.996+0000 7fa15f800640 1 -- 172.21.15.32:0/1541051172 wait complete. 2024-04-12T19:50:32.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91000cd10 con 0x7fc918054690 2024-04-12T19:50:32.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fa15f800640 1 Processor -- start 2024-04-12T19:50:32.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fc925200640 1 -- 172.21.15.32:0/1798411770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91804d620 con 0x7fc918054690 2024-04-12T19:50:32.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fc76de00640 1 Processor -- start 2024-04-12T19:50:32.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.000+0000 7fc76de00640 1 -- start start 2024-04-12T19:50:32.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:31.999+0000 7fa15f800640 1 -- start start 2024-04-12T19:50:32.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 0x7fa158161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fc76de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc760050dd0 0x7fc760038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.002+0000 7fc76de00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76004f540 con 0x7fc76003a5d0 2024-04-12T19:50:32.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.002+0000 7fc76de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76004f820 con 0x7fc7600e1090 2024-04-12T19:50:32.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.002+0000 7fc76de00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc76004fb00 con 0x7fc760050dd0 2024-04-12T19:50:32.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.002+0000 7fc925200640 1 -- 172.21.15.32:0/1798411770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91804dbe0 con 0x7fc918054690 2024-04-12T19:50:32.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa15806fca0 con 0x7fa1580b4970 2024-04-12T19:50:32.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa1580b3650 con 0x7fa158162df0 2024-04-12T19:50:32.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.001+0000 7fa15f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1580b38a0 con 0x7fa158072640 2024-04-12T19:50:32.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fc76ca00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc7600e1090 0x7fc760039ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:55664/0 (socket says 172.21.15.32:55664) 2024-04-12T19:50:32.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fc76ca00640 1 -- 172.21.15.32:0/463904419 learned_addr learned my addr 172.21.15.32:0/463904419 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:32.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fc76d400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc76003a5d0 0x7fc760005f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43276/0 (socket says 172.21.15.32:43276) 2024-04-12T19:50:32.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7f6302e00640 1 Processor -- start 2024-04-12T19:50:32.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc925200640 1 -- 172.21.15.32:0/1798411770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc918008a00 con 0x7fc918054690 2024-04-12T19:50:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fa157e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fa15d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 0x7fa158161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fa15ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fa15ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:55012/0 (socket says 172.21.15.32:55012) 2024-04-12T19:50:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.003+0000 7fa15ce00640 1 -- 172.21.15.32:0/4054137675 learned_addr learned my addr 172.21.15.32:0/4054137675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767e00640 1 --2- 172.21.15.32:0/463904419 >> v2:172.21.15.32:3300/0 conn(0x7fc760050dd0 0x7fc760038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562364332 0 0) 0x7fc76004f540 con 0x7fc76003a5d0 2024-04-12T19:50:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc750003520 con 0x7fc76003a5d0 2024-04-12T19:50:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7f6302e00640 1 -- start start 2024-04-12T19:50:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851585294 0 0) 0x7fc76004f820 con 0x7fc7600e1090 2024-04-12T19:50:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc76004f540 con 0x7fc7600e1090 2024-04-12T19:50:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767e00640 1 -- 172.21.15.32:0/463904419 >> v1:172.21.15.169:6789/0 conn(0x7fc7600e1090 legacy=0x7fc760039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fc767e00640 1 -- 172.21.15.32:0/463904419 >> v1:172.21.15.32:6789/0 conn(0x7fc76003a5d0 legacy=0x7fc760005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7f9225e00640 1 Processor -- start 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc91001a070 con 0x7fc918054690 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fa15ce00640 1 -- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 msgr2=0x7fa158161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fa15ce00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 0x7fa158161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.004+0000 7fa15ce00640 1 -- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 msgr2=0x7fa1580ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fa15ce00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fa15ce00640 1 -- 172.21.15.32:0/4054137675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1580ae050 con 0x7fa158072640 2024-04-12T19:50:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7f9225e00640 1 -- start start 2024-04-12T19:50:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fc767e00640 1 -- 172.21.15.32:0/463904419 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76004fde0 con 0x7fc760050dd0 2024-04-12T19:50:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fa15d800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 0x7fa158161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.005+0000 7fa157e00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7fa15ce00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c7b0 tx=0x7fa14000cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7fc916000640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 0x7fc90006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f6302e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f62fc148210 0x7f62fc0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f6302e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc0afcf0 0x7f62fc072970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f6302e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62fc0b3970 con 0x7f62fc072eb0 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f6302e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f62fc0b42a0 con 0x7f62fc0afcf0 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f6302e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f62fc0b3c50 con 0x7f62fc148210 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f62fb400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc0afcf0 0x7f62fc072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f62fb400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62fc0afcf0 0x7f62fc072970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32936/0 (socket says 172.21.15.32:32936) 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f62fb400640 1 -- 172.21.15.32:0/1168888962 learned_addr learned my addr 172.21.15.32:0/1168888962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.006+0000 7f7963400640 1 Processor -- start 2024-04-12T19:50:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f7963400640 1 -- start start 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92200d2c30 0x7f92200ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92200d1300 0x7f92200d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f92200d3090 con 0x7f92200d2c30 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f92200ba8d0 con 0x7f92200d1300 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7f9225e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92200d32c0 con 0x7f922015ec40 2024-04-12T19:50:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7fc767e00640 1 --2- 172.21.15.32:0/463904419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc760038630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0097a0 tx=0x7fc75c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d7b11f70001fea server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140018070 con 0x7fa158072640 2024-04-12T19:50:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1400074e0 con 0x7fa158072640 2024-04-12T19:50:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.007+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140004a20 con 0x7fa158072640 2024-04-12T19:50:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f62fb400640 1 -- 172.21.15.32:0/1168888962 >> v2:172.21.15.169:3300/0 conn(0x7f62fc148210 msgr2=0x7f62fc0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f62fb400640 1 --2- 172.21.15.32:0/1168888962 >> v2:172.21.15.169:3300/0 conn(0x7f62fc148210 0x7f62fc0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1580a96b0 con 0x7fa158072640 2024-04-12T19:50:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f921e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92200d2c30 0x7f92200ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f921e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92200d2c30 0x7f92200ba390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32944/0 (socket says 172.21.15.32:32944) 2024-04-12T19:50:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f921e000640 1 -- 172.21.15.32:0/58180696 learned_addr learned my addr 172.21.15.32:0/58180696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f921ea00640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f921f400640 1 --2- 172.21.15.32:0/58180696 >> v2:172.21.15.169:3300/0 conn(0x7f92200d1300 0x7f92200d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc910057110 con 0x7fc918054690 2024-04-12T19:50:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7fc91de00640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 0x7fc90006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1580a9c70 con 0x7fa158072640 2024-04-12T19:50:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f62fb400640 1 -- 172.21.15.32:0/1168888962 >> v1:172.21.15.32:6789/0 conn(0x7f62fc072eb0 legacy=0x7f62fc15ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.008+0000 7f62fb400640 1 -- 172.21.15.32:0/1168888962 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc0b3f30 con 0x7f62fc0afcf0 2024-04-12T19:50:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.010+0000 7fc91de00640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 0x7fc90006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc920075df0 tx=0x7fc920063490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.010+0000 7f62fb400640 1 --2- 172.21.15.32:0/1168888962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc0afcf0 0x7f62fc072970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0b4c70 tx=0x7f62e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01b7687d59a4b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f7963400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f795c15aaa0 0x7f795c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.009+0000 7f921e000640 1 -- 172.21.15.32:0/58180696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 msgr2=0x7f92200d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.009+0000 7f921e000640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.010+0000 7fa14e000640 1 -- 172.21.15.32:0/4054137675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1580755c0 con 0x7fa158072640 2024-04-12T19:50:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.009+0000 7f921e000640 1 -- 172.21.15.32:0/58180696 >> v2:172.21.15.169:3300/0 conn(0x7f92200d1300 msgr2=0x7f92200d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.009+0000 7f921e000640 1 --2- 172.21.15.32:0/58180696 >> v2:172.21.15.169:3300/0 conn(0x7f92200d1300 0x7f92200d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.009+0000 7f921e000640 1 -- 172.21.15.32:0/58180696 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92200d35a0 con 0x7f92200d2c30 2024-04-12T19:50:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f921e000640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d2c30 0x7f92200ba390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f922008dde0 tx=0x7f921802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb430cdd5135cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f7960a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f795c15aaa0 0x7f795c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f7960a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f795c15aaa0 0x7f795c0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32954/0 (socket says 172.21.15.32:32954) 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f7960a00640 1 -- 172.21.15.32:0/946891078 learned_addr learned my addr 172.21.15.32:0/946891078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.011+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795c15ae80 con 0x7f795c0b3ca0 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.012+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795c0b41e0 con 0x7f795c072810 2024-04-12T19:50:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.012+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f795c0b44c0 con 0x7f795c15aaa0 2024-04-12T19:50:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.012+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa14001b070 con 0x7fa158072640 2024-04-12T19:50:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.012+0000 7f795b400640 1 -- 172.21.15.32:0/946891078 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882907389 0 0) 0x7f795c0b41e0 con 0x7f795c072810 2024-04-12T19:50:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.012+0000 7f795b400640 1 -- 172.21.15.32:0/946891078 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7940003520 con 0x7f795c072810 2024-04-12T19:50:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.013+0000 7f7960a00640 1 -- 172.21.15.32:0/946891078 >> v1:172.21.15.169:6789/0 conn(0x7f795c072810 legacy=0x7f795c0cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.013+0000 7f7960a00640 1 -- 172.21.15.32:0/946891078 >> v1:172.21.15.32:6789/0 conn(0x7f795c0b3ca0 legacy=0x7f795c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.013+0000 7f7960a00640 1 -- 172.21.15.32:0/946891078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c0b47a0 con 0x7f795c15aaa0 2024-04-12T19:50:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.013+0000 7fefd1400640 1 Processor -- start 2024-04-12T19:50:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.014+0000 7fa154c00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 0x7fa13406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.014+0000 7fefd1400640 1 -- start start 2024-04-12T19:50:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.014+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa14008b1b0 con 0x7fa158072640 2024-04-12T19:50:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.014+0000 7fa157e00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 0x7fa13406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f7960a00640 1 --2- 172.21.15.32:0/946891078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c15aaa0 0x7f795c0b3760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f795400ca00 tx=0x7f795402ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac66b22862dcf73d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f921d600640 1 -- 172.21.15.32:0/58180696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218034070 con 0x7f92200d2c30 2024-04-12T19:50:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f921d600640 1 -- 172.21.15.32:0/58180696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9218030610 con 0x7f92200d2c30 2024-04-12T19:50:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f795b400640 1 -- 172.21.15.32:0/946891078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954034070 con 0x7f795c15aaa0 2024-04-12T19:50:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f921d600640 1 -- 172.21.15.32:0/58180696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218030a20 con 0x7f92200d2c30 2024-04-12T19:50:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f795b400640 1 -- 172.21.15.32:0/946891078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79540094e0 con 0x7f795c15aaa0 2024-04-12T19:50:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7f795b400640 1 -- 172.21.15.32:0/946891078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540098b0 con 0x7f795c15aaa0 2024-04-12T19:50:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.015+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c035020 con 0x7fc760050dd0 2024-04-12T19:50:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7f62faa00640 1 -- 172.21.15.32:0/1168888962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4035020 con 0x7f62fc0afcf0 2024-04-12T19:50:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc910059070 con 0x7fc918054690 2024-04-12T19:50:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fefd1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 0x7fefcc0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fefd1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fefcc06ffe0 0x7fefcc0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fefd1400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefcc071330 con 0x7fefcc15e8d0 2024-04-12T19:50:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fefd1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fefcc070430 con 0x7fefcc06ffe0 2024-04-12T19:50:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fefd1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc0b47a0 con 0x7fefcc0b06b0 2024-04-12T19:50:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7f62faa00640 1 -- 172.21.15.32:0/1168888962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4031870 con 0x7f62fc0afcf0 2024-04-12T19:50:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c031830 con 0x7fc760050dd0 2024-04-12T19:50:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7fc767400640 1 -- 172.21.15.32:0/463904419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c031c00 con 0x7fc760050dd0 2024-04-12T19:50:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7f62faa00640 1 -- 172.21.15.32:0/1168888962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4031c40 con 0x7f62fc0afcf0 2024-04-12T19:50:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c15aaa0 msgr2=0x7f795c0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.016+0000 7f7963400640 1 --2- 172.21.15.32:0/946891078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c15aaa0 0x7f795c0b3760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f795400ca00 tx=0x7f795402ca00 comp rx=0 tx=0).stop 2024-04-12T19:50:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fefca800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fefcc15e8d0 0x7fefcc0b0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:55672/0 (socket says 172.21.15.32:55672) 2024-04-12T19:50:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 -- 172.21.15.32:0/463904419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 msgr2=0x7fc760038630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 --2- 172.21.15.32:0/463904419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc760038630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0097a0 tx=0x7fc75c02b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fefca800640 1 -- 172.21.15.32:0/1832576668 learned_addr learned my addr 172.21.15.32:0/1832576668 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 -- 172.21.15.32:0/463904419 shutdown_connections 2024-04-12T19:50:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 --2- 172.21.15.32:0/463904419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc760038630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fefc9e00640 1 --2- 172.21.15.32:0/1832576668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 0x7fefcc0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fefcb200640 1 --2- 172.21.15.32:0/1832576668 >> v2:172.21.15.169:3300/0 conn(0x7fefcc06ffe0 0x7fefcc0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fa157e00640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 0x7fa13406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa15815e420 tx=0x7fa150009550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 -- 172.21.15.32:0/1168888962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc0afcf0 msgr2=0x7f62fc072970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 --2- 172.21.15.32:0/1168888962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc0afcf0 0x7f62fc072970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0b4c70 tx=0x7f62e402b910 comp rx=0 tx=0).stop 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 shutdown_connections 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f7963400640 1 --2- 172.21.15.32:0/946891078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c15aaa0 0x7f795c0b3760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 >> 172.21.15.32:0/946891078 conn(0x7f795c0b2460 msgr2=0x7f795c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 -- 172.21.15.32:0/463904419 >> 172.21.15.32:0/463904419 conn(0x7fc760009fb0 msgr2=0x7fc76000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f9225e00640 1 -- 172.21.15.32:0/58180696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d2c30 msgr2=0x7f92200ba390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f9225e00640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d2c30 0x7f92200ba390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f922008dde0 tx=0x7f921802b960 comp rx=0 tx=0).stop 2024-04-12T19:50:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fb259c00640 1 Processor -- start 2024-04-12T19:50:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7fc76de00640 1 -- 172.21.15.32:0/463904419 shutdown_connections 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 -- 172.21.15.32:0/1168888962 shutdown_connections 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 --2- 172.21.15.32:0/1168888962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc0afcf0 0x7f62fc072970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 --2- 172.21.15.32:0/1168888962 >> v2:172.21.15.169:3300/0 conn(0x7f62fc148210 0x7f62fc0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f6302e00640 1 -- 172.21.15.32:0/1168888962 >> 172.21.15.32:0/1168888962 conn(0x7f62fc0b28b0 msgr2=0x7f62fc0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 shutdown_connections 2024-04-12T19:50:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.017+0000 7f7963400640 1 -- 172.21.15.32:0/946891078 wait complete. 2024-04-12T19:50:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fb259c00640 1 -- start start 2024-04-12T19:50:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fc76de00640 1 -- 172.21.15.32:0/463904419 wait complete. 2024-04-12T19:50:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f6302e00640 1 -- 172.21.15.32:0/1168888962 shutdown_connections 2024-04-12T19:50:32.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f9225e00640 1 -- 172.21.15.32:0/58180696 shutdown_connections 2024-04-12T19:50:32.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f9225e00640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d2c30 0x7f92200ba390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f9225e00640 1 --2- 172.21.15.32:0/58180696 >> v2:172.21.15.169:3300/0 conn(0x7f92200d1300 0x7f92200d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f9225e00640 1 --2- 172.21.15.32:0/58180696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f9225e00640 1 -- 172.21.15.32:0/58180696 >> 172.21.15.32:0/58180696 conn(0x7f9220073690 msgr2=0x7f922006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fefc9e00640 1 -- 172.21.15.32:0/1832576668 >> v2:172.21.15.169:3300/0 conn(0x7fefcc06ffe0 msgr2=0x7fefcc0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fefc9e00640 1 --2- 172.21.15.32:0/1832576668 >> v2:172.21.15.169:3300/0 conn(0x7fefcc06ffe0 0x7fefcc0b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fefc9e00640 1 -- 172.21.15.32:0/1832576668 >> v1:172.21.15.169:6789/0 conn(0x7fefcc15e8d0 legacy=0x7fefcc0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7fefc9e00640 1 -- 172.21.15.32:0/1832576668 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc0b4a80 con 0x7fefcc0b06b0 2024-04-12T19:50:32.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.018+0000 7f6302e00640 1 -- 172.21.15.32:0/1168888962 wait complete. 2024-04-12T19:50:32.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.019+0000 7f9225e00640 1 -- 172.21.15.32:0/58180696 shutdown_connections 2024-04-12T19:50:32.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.019+0000 7f9225e00640 1 -- 172.21.15.32:0/58180696 wait complete. 2024-04-12T19:50:32.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7f6302e00640 1 Processor -- start 2024-04-12T19:50:32.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7f6302e00640 1 -- start start 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7fb259c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb25415ec60 0x7fb2540ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7fb259c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2540d3c90 0x7fb2540b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7fb259c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2540b8f30 con 0x7fb2540b63d0 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7fb259c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb2540b9210 con 0x7fb25415ec60 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7fb259c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb2540b94f0 con 0x7fb2540d3c90 2024-04-12T19:50:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.020+0000 7f7963400640 1 Processor -- start 2024-04-12T19:50:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fb252000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb2540b63d0 0x7fb2540b7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43292/0 (socket says 172.21.15.32:43292) 2024-04-12T19:50:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fb252000640 1 -- 172.21.15.32:0/3166899989 learned_addr learned my addr 172.21.15.32:0/3166899989 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fefc9e00640 1 --2- 172.21.15.32:0/1832576668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 0x7fefcc0ae5e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc9cc3395d60f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fc76de00640 1 Processor -- start 2024-04-12T19:50:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fc76de00640 1 -- start start 2024-04-12T19:50:32.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7f7963400640 1 -- start start 2024-04-12T19:50:32.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7f6302e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 0x7f62fc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7f6302e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 0x7f62fc143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7f6302e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc143670 0x7f62fc15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7f6302e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc148770 con 0x7f62fc072eb0 2024-04-12T19:50:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.021+0000 7fb252a00640 1 --2- 172.21.15.32:0/3166899989 >> v2:172.21.15.169:3300/0 conn(0x7fb25415ec60 0x7fb2540ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f62fb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 0x7f62fc143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f9225e00640 1 Processor -- start 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f62fb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 0x7f62fc143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51756/0 (socket says 172.21.15.32:51756) 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f62fb400640 1 -- 172.21.15.32:0/3581093757 learned_addr learned my addr 172.21.15.32:0/3581093757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f6302e00640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc1485f0 con 0x7f62fc148210 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f6302e00640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f62fc070760 con 0x7f62fc143670 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f62fbe00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 0x7f62fc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb253400640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2540d3c90 0x7fb2540b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182458352 0 0) 0x7fb2540b8f30 con 0x7fb2540b63d0 2024-04-12T19:50:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb238003520 con 0x7fb2540b63d0 2024-04-12T19:50:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f6300e00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc143670 0x7f62fc15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7f9225e00640 1 -- start start 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2778676170 0 0) 0x7fb238003520 con 0x7fb2540b63d0 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2540d3c90 msgr2=0x7fb2540b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2540d3c90 0x7fb2540b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 >> v2:172.21.15.169:3300/0 conn(0x7fb25415ec60 msgr2=0x7fb2540ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 --2- 172.21.15.32:0/3166899989 >> v2:172.21.15.169:3300/0 conn(0x7fb25415ec60 0x7fb2540ba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2540b97d0 con 0x7fb2540b63d0 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fb240001670 con 0x7fb2540b63d0 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 >> v1:172.21.15.32:6789/0 conn(0x7fb2540b63d0 legacy=0x7fb2540b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.022+0000 7fb251600640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2380039b0 0x7fb238003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb251600640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb238004800 0x7fb238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb251600640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2380051c0 0x7fb23800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540b9210 con 0x7fb2380051c0 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540b94f0 con 0x7fb2380039b0 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb23800c0e0 con 0x7fb238004800 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 0x7fc7600df990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 -- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc143670 msgr2=0x7f62fc15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc143670 0x7f62fc15ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 -- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 msgr2=0x7f62fc143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 0x7f62fc143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc15e970 con 0x7f62fc072eb0 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 0x7fc76004ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600066a0 con 0x7fc760050dd0 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600503a0 con 0x7fc7600e1090 2024-04-12T19:50:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fc76de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc760006820 con 0x7fc76003a5d0 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 0x7f795c14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f795c15aaa0 0x7f795c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f795c06f770 con 0x7f795c072810 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0ceba0 con 0x7f795c0b3ca0 2024-04-12T19:50:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f7963400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f795c0701f0 con 0x7f795c15aaa0 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7f62fbe00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 0x7f62fc15e430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62f400bfc0 tx=0x7f62f4009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc76ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 0x7fc7600df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc76d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 0x7fc76004ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc767e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc767e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32982/0 (socket says 172.21.15.32:32982) 2024-04-12T19:50:32.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc767e00640 1 -- 172.21.15.32:0/971871138 learned_addr learned my addr 172.21.15.32:0/971871138 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f795be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f795be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51776/0 (socket says 172.21.15.32:51776) 2024-04-12T19:50:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f795be00640 1 -- 172.21.15.32:0/1212429962 learned_addr learned my addr 172.21.15.32:0/1212429962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f7960a00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 0x7f795c14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4018070 con 0x7f62fc072eb0 2024-04-12T19:50:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f6302e00640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc1f8010 con 0x7f62fc072eb0 2024-04-12T19:50:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.023+0000 7fb253400640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2380051c0 0x7fb23800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb252a00640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2380039b0 0x7fb238003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb252000640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb238004800 0x7fb238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb253400640 1 -- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb238004800 msgr2=0x7fb238004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb253400640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb238004800 0x7fb238004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb253400640 1 -- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2380039b0 msgr2=0x7fb238003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb253400640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2380039b0 0x7fb238003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fb253400640 1 -- 172.21.15.32:0/3166899989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb238003520 con 0x7fb2380051c0 2024-04-12T19:50:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7fc767e00640 1 -- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 msgr2=0x7fc7600df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f6302e00640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1f85d0 con 0x7f62fc072eb0 2024-04-12T19:50:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc767e00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 0x7fc7600df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc767e00640 1 -- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 msgr2=0x7fc76004ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc767e00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 0x7fc76004ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc767e00640 1 -- 172.21.15.32:0/971871138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760179e00 con 0x7fc760050dd0 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.024+0000 7f7961400640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f795c15aaa0 0x7f795c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f795be00640 1 -- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f795c15aaa0 msgr2=0x7f795c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f795be00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f795c15aaa0 0x7f795c15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f795be00640 1 -- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 msgr2=0x7f795c14f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f9225e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 0x7f9220161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc76d400640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 0x7fc76004ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc76ca00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 0x7fc7600df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f4013620 con 0x7f62fc072eb0 2024-04-12T19:50:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4013a10 con 0x7f62fc072eb0 2024-04-12T19:50:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7fc767e00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098c0 tx=0x7fc75c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.025+0000 7f9225e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f9225e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c048070 con 0x7fc760050dd0 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f9225e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92200d3e40 con 0x7f92200d1300 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f9225e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92200d3b60 con 0x7f92200d2c30 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f9225e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92200b63b0 con 0x7f922015ec40 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c03a6f0 con 0x7fc760050dd0 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c03aae0 con 0x7fc760050dd0 2024-04-12T19:50:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76017a0a0 con 0x7fc760050dd0 2024-04-12T19:50:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f921e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76017a660 con 0x7fc760050dd0 2024-04-12T19:50:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f921e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:51794/0 (socket says 172.21.15.32:51794) 2024-04-12T19:50:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f921e000640 1 -- 172.21.15.32:0/2175356594 learned_addr learned my addr 172.21.15.32:0/2175356594 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f795be00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 0x7f795c14f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f795be00640 1 -- 172.21.15.32:0/1212429962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c152780 con 0x7f795c0b3ca0 2024-04-12T19:50:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.026+0000 7f921f400640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921ea00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 0x7f9220161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f7960a00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 0x7f795c14f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc143a80 con 0x7f62fc072eb0 2024-04-12T19:50:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f795be00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c470 tx=0x7f794c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921e000640 1 -- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 msgr2=0x7f92200ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921e000640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921e000640 1 -- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 msgr2=0x7f9220161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921e000640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 0x7f9220161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921e000640 1 -- 172.21.15.32:0/2175356594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92200cbee0 con 0x7f92200d2c30 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7f921ea00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 0x7f9220161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.027+0000 7fb253400640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2380051c0 0x7fb23800b9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2540d4070 tx=0x7fb238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bd44fa7b037a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c039070 con 0x7fb2380051c0 2024-04-12T19:50:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0052d0 con 0x7f795c0b3ca0 2024-04-12T19:50:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc760005390 con 0x7fc760050dd0 2024-04-12T19:50:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7fb251600640 1 -- 172.21.15.32:0/3166899989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c02cb90 con 0x7fb2380051c0 2024-04-12T19:50:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c005470 con 0x7f795c0b3ca0 2024-04-12T19:50:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c007260 con 0x7f795c0b3ca0 2024-04-12T19:50:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7f921e000640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9218000c00 tx=0x7f9218004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.028+0000 7f7963400640 1 -- 172.21.15.32:0/1212429962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c14ff30 con 0x7f795c0b3ca0 2024-04-12T19:50:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7f7963400640 1 -- 172.21.15.32:0/1212429962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1504f0 con 0x7f795c0b3ca0 2024-04-12T19:50:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb259c00640 1 -- 172.21.15.32:0/3166899989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2380051c0 msgr2=0x7fb23800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb259c00640 1 --2- 172.21.15.32:0/3166899989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2380051c0 0x7fb23800b9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2540d4070 tx=0x7fb238005830 comp rx=0 tx=0).stop 2024-04-12T19:50:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb253400640 1 -- 172.21.15.32:0/3166899989 reap_dead start 2024-04-12T19:50:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb259c00640 1 -- 172.21.15.32:0/3166899989 shutdown_connections 2024-04-12T19:50:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb259c00640 1 -- 172.21.15.32:0/3166899989 >> 172.21.15.32:0/3166899989 conn(0x7fb254073480 msgr2=0x7fb254073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218048070 con 0x7f92200d2c30 2024-04-12T19:50:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9218030410 con 0x7f92200d2c30 2024-04-12T19:50:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7fb259c00640 1 -- 172.21.15.32:0/3166899989 shutdown_connections 2024-04-12T19:50:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.030+0000 7fb259c00640 1 -- 172.21.15.32:0/3166899989 wait complete. 2024-04-12T19:50:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62f401b070 con 0x7f62fc072eb0 2024-04-12T19:50:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.029+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218034040 con 0x7f92200d2c30 2024-04-12T19:50:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.030+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92200cc1c0 con 0x7f92200d2c30 2024-04-12T19:50:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.030+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92200cca60 con 0x7f92200d2c30 2024-04-12T19:50:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.031+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc75c04b070 con 0x7fc760050dd0 2024-04-12T19:50:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.031+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92200cc390 con 0x7f92200d2c30 2024-04-12T19:50:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.031+0000 7f62f3e00640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.032+0000 7fb259c00640 1 Processor -- start 2024-04-12T19:50:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.032+0000 7fb259c00640 1 -- start start 2024-04-12T19:50:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.031+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f62f408bdf0 con 0x7f62fc072eb0 2024-04-12T19:50:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.032+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c068c80 con 0x7f795c0b3ca0 2024-04-12T19:50:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.032+0000 7fc764c00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 0x7fc74806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7f62fb400640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc75c0796f0 con 0x7fc760050dd0 2024-04-12T19:50:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2380051c0 0x7fb254158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2540cde80 0x7fb2540ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540ba7b0 con 0x7fb2540cba20 2024-04-12T19:50:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb25415f040 con 0x7fb2540cde80 2024-04-12T19:50:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fb259c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb2540ba930 con 0x7fb2380051c0 2024-04-12T19:50:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.033+0000 7fc76ca00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 0x7fc74806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb252a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2380051c0 0x7fb254158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33018/0 (socket says 172.21.15.32:33018) 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 -- 172.21.15.32:0/2129743211 learned_addr learned my addr 172.21.15.32:0/2129743211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb252000640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2540cde80 0x7fb2540ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fefc9400640 1 -- 172.21.15.32:0/1832576668 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8031780 con 0x7fefcc0b06b0 2024-04-12T19:50:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fefc9400640 1 -- 172.21.15.32:0/1832576668 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb8037070 con 0x7fefcc0b06b0 2024-04-12T19:50:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fefc9400640 1 -- 172.21.15.32:0/1832576668 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8031b40 con 0x7fefcc0b06b0 2024-04-12T19:50:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7f62fb400640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 0x7f62d806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62fc1600a0 tx=0x7f62e403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 -- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2380051c0 msgr2=0x7fb254158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2380051c0 0x7fb254158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 -- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2540cde80 msgr2=0x7fb2540ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2540cde80 0x7fb2540ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.034+0000 7fb253400640 1 -- 172.21.15.32:0/2129743211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2540cc750 con 0x7fb2540cba20 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.035+0000 7fb253400640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009f80 tx=0x7fb24c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.035+0000 7fefd1400640 1 -- 172.21.15.32:0/1832576668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 msgr2=0x7fefcc0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.035+0000 7fefd1400640 1 --2- 172.21.15.32:0/1832576668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 0x7fefcc0ae5e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fefb80097a0 tx=0x7fefb802b820 comp rx=0 tx=0).stop 2024-04-12T19:50:32.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 -- 172.21.15.32:0/1832576668 shutdown_connections 2024-04-12T19:50:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 --2- 172.21.15.32:0/1832576668 >> v2:172.21.15.169:3300/0 conn(0x7fefcc06ffe0 0x7fefcc0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 --2- 172.21.15.32:0/1832576668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc0b06b0 0x7fefcc0ae5e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 -- 172.21.15.32:0/1832576668 >> 172.21.15.32:0/1832576668 conn(0x7fefcc0b6b20 msgr2=0x7fefcc0b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9218037070 con 0x7f92200d2c30 2024-04-12T19:50:32.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 -- 172.21.15.32:0/1832576668 shutdown_connections 2024-04-12T19:50:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fefd1400640 1 -- 172.21.15.32:0/1832576668 wait complete. 2024-04-12T19:50:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1581631d0 con 0x7fa158072640 2024-04-12T19:50:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.035+0000 7fc76ca00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 0x7fc74806ee30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc76004d640 tx=0x7fc754009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.037+0000 7f9217400640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 0x7f91fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.037+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f92180ad750 con 0x7f92200d2c30 2024-04-12T19:50:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.036+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f794c007400 con 0x7f795c0b3ca0 2024-04-12T19:50:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.037+0000 7f7958c00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.038+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f794c08b0e0 con 0x7f795c0b3ca0 2024-04-12T19:50:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.038+0000 7f7960a00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.037+0000 7f921ea00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 0x7f91fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.038+0000 7fefd1400640 1 Processor -- start 2024-04-12T19:50:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.039+0000 7f921ea00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 0x7f91fc06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.039+0000 7fefd1400640 1 -- start start 2024-04-12T19:50:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.039+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc75c052020 con 0x7fc760050dd0 2024-04-12T19:50:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.039+0000 7f7960a00640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7954036000 tx=0x7f7954009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefcc0b06b0 0x7fefcc14d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefcc15e8d0 0x7fefcc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc15ecb0 con 0x7fefcc15e8d0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc071050 con 0x7fefcc0b06b0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.040+0000 7fefd1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc15ee30 con 0x7fefcc06ffe0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.041+0000 7fefc9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefcc0b06b0 0x7fefcc14d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.041+0000 7fefca800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.041+0000 7fefca800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:55104/0 (socket says 172.21.15.32:55104) 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.041+0000 7fefca800640 1 -- 172.21.15.32:0/1918162088 learned_addr learned my addr 172.21.15.32:0/1918162088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.041+0000 7fefcb200640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefcc15e8d0 0x7fefcc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.042+0000 7fefca800640 1 -- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefcc0b06b0 msgr2=0x7fefcc14d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.042+0000 7fefca800640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefcc0b06b0 0x7fefcc14d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.042+0000 7fefca800640 1 -- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefcc15e8d0 msgr2=0x7fefcc14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.042+0000 7fefca800640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefcc15e8d0 0x7fefcc14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.042+0000 7fefca800640 1 -- 172.21.15.32:0/1918162088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc15f920 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.043+0000 7fefca800640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c4a0 tx=0x7fefbc00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.043+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc017070 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.043+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc005ce0 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.043+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc007270 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.044+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc0b3d10 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.044+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc15fbc0 con 0x7fefcc06ffe0 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.047+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c039070 con 0x7fb2540cba20 2024-04-12T19:50:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.047+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c044070 con 0x7fb2540cba20 2024-04-12T19:50:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.047+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c03e850 con 0x7fb2540cba20 2024-04-12T19:50:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.047+0000 7fb259c00640 1 -- 172.21.15.32:0/2129743211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2540cca30 con 0x7fb2540cba20 2024-04-12T19:50:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.047+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62fc15e970 con 0x7f62fc072eb0 2024-04-12T19:50:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.048+0000 7fb259c00640 1 -- 172.21.15.32:0/2129743211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2540ccff0 con 0x7fb2540cba20 2024-04-12T19:50:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.048+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc068c80 con 0x7fefcc06ffe0 2024-04-12T19:50:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.050+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb228005290 con 0x7fb2540cba20 2024-04-12T19:50:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.049+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefbc01a070 con 0x7fefcc06ffe0 2024-04-12T19:50:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.052+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9218051020 con 0x7f92200d2c30 2024-04-12T19:50:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.052+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb24c02cb90 con 0x7fb2540cba20 2024-04-12T19:50:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.052+0000 7fefc2a00640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.052+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fefbc08ada0 con 0x7fefcc06ffe0 2024-04-12T19:50:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.053+0000 7fefc9e00640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.054+0000 7fb24aa00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 0x7fb23406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.054+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb24c0adaf0 con 0x7fb2540cba20 2024-04-12T19:50:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.054+0000 7fb252a00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 0x7fb23406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.055+0000 7fefc9e00640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 0x7fefac06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefb8036000 tx=0x7fefb803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.055+0000 7fb252a00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 0x7fb23406ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb244006fd0 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.062+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb24c07b940 con 0x7fb2540cba20 2024-04-12T19:50:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.063+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefbc058b70 con 0x7fefcc06ffe0 2024-04-12T19:50:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.067+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f795c15ae80 con 0x7f795c0b3ca0 2024-04-12T19:50:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.446+0000 7fc925200640 1 -- 172.21.15.32:0/1798411770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc918037980 con 0x7fc918054690 2024-04-12T19:50:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.447+0000 7fc916000640 1 -- 172.21.15.32:0/1798411770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc918037980 con 0x7fc918054690 2024-04-12T19:50:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.451+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 msgr2=0x7fc90006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.451+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 0x7fc90006ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc920075df0 tx=0x7fc920063490 comp rx=0 tx=0).stop 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.451+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 msgr2=0x7fc91804a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.451+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc91000e850 tx=0x7fc91000ed10 comp rx=0 tx=0).stop 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 shutdown_connections 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc90006c940 0x7fc90006ee00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9180df130 0x7fc91804ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc918054690 0x7fc91804a7d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 --2- 172.21.15.32:0/1798411770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc91800aad0 0x7fc918185ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 >> 172.21.15.32:0/1798411770 conn(0x7fc918009fe0 msgr2=0x7fc9180dbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.452+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 shutdown_connections 2024-04-12T19:50:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.453+0000 7fc90b400640 1 -- 172.21.15.32:0/1798411770 wait complete. 2024-04-12T19:50:32.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.468+0000 7fa14e000640 1 -- 172.21.15.32:0/4054137675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa158160cb0 con 0x7fa158072640 2024-04-12T19:50:32.473 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-12T19:50:32.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.469+0000 7fa154c00640 1 -- 172.21.15.32:0/4054137675 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa140059650 con 0x7fa158072640 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 msgr2=0x7fa13406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 0x7fa13406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa15815e420 tx=0x7fa150009550 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 msgr2=0x7fa1581515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c7b0 tx=0x7fa14000cc70 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 shutdown_connections 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa13406c940 0x7fa13406ee00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa158162df0 0x7fa158161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.473+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1580b4970 0x7fa1580ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.474+0000 7fa15f800640 1 --2- 172.21.15.32:0/4054137675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa158072640 0x7fa1581515f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.474+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 >> 172.21.15.32:0/4054137675 conn(0x7fa15806b930 msgr2=0x7fa15806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.474+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 shutdown_connections 2024-04-12T19:50:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.474+0000 7fa15f800640 1 -- 172.21.15.32:0/4054137675 wait complete. 2024-04-12T19:50:32.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2024-04-12T19:50:32.479 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.481+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc7600dfed0 con 0x7fc760050dd0 2024-04-12T19:50:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.481+0000 7fc764c00640 1 -- 172.21.15.32:0/971871138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc75c031800 con 0x7fc760050dd0 2024-04-12T19:50:32.482 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 msgr2=0x7fc74806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 0x7fc74806ee30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc76004d640 tx=0x7fc754009040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 msgr2=0x7fc76004a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098c0 tx=0x7fc75c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 shutdown_connections 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc74806c970 0x7fc74806ee30 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc76004d640 tx=0x7fc754009040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7600e1090 0x7fc76004ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc760050dd0 0x7fc76004a7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 --2- 172.21.15.32:0/971871138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc76003a5d0 0x7fc7600df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.485+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 >> 172.21.15.32:0/971871138 conn(0x7fc760009fb0 msgr2=0x7fc760006bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.486+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f92200c7db0 con 0x7f92200d2c30 2024-04-12T19:50:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.486+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 shutdown_connections 2024-04-12T19:50:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.486+0000 7fc76de00640 1 -- 172.21.15.32:0/971871138 wait complete. 2024-04-12T19:50:32.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.487+0000 7f9217400640 1 -- 172.21.15.32:0/2175356594 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92200c7db0 con 0x7f92200d2c30 2024-04-12T19:50:32.487 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-12T19:50:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 msgr2=0x7f91fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 0x7f91fc06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000b040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 msgr2=0x7f92200c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9218000c00 tx=0x7f9218004730 comp rx=0 tx=0).stop 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 shutdown_connections 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f91fc06c8f0 0x7f91fc06edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f922015ec40 0x7f92200ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92200d2c30 0x7f92200c9b10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 --2- 172.21.15.32:0/2175356594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92200d1300 0x7f9220161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.490+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 >> 172.21.15.32:0/2175356594 conn(0x7f9220073690 msgr2=0x7f922006ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.491+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 shutdown_connections 2024-04-12T19:50:32.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.491+0000 7f9225e00640 1 -- 172.21.15.32:0/2175356594 wait complete. 2024-04-12T19:50:32.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.499+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f62fc068c80 con 0x7f62fc072eb0 2024-04-12T19:50:32.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.500+0000 7f62f3e00640 1 -- 172.21.15.32:0/3581093757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62f405a210 con 0x7f62fc072eb0 2024-04-12T19:50:32.500 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.503+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 msgr2=0x7f62d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.503+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 0x7f62d806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62fc1600a0 tx=0x7f62e403c040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.503+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 msgr2=0x7f62fc15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.503+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 0x7f62fc15e430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62f400bfc0 tx=0x7f62f4009ef0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 shutdown_connections 2024-04-12T19:50:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f62d806c8f0 0x7f62d806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc143670 0x7f62fc15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc148210 0x7f62fc143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 --2- 172.21.15.32:0/3581093757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62fc072eb0 0x7f62fc15e430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 >> 172.21.15.32:0/3581093757 conn(0x7f62fc0b28b0 msgr2=0x7f62fc0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-04-12T19:50:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 shutdown_connections 2024-04-12T19:50:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.504+0000 7f62f1600640 1 -- 172.21.15.32:0/3581093757 wait complete. 2024-04-12T19:50:32.506 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.506+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb228005580 con 0x7fb2540cba20 2024-04-12T19:50:32.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.507+0000 7fb24aa00640 1 -- 172.21.15.32:0/2129743211 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb24c07fc40 con 0x7fb2540cba20 2024-04-12T19:50:32.507 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-12T19:50:32.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-04-12T19:50:32.511 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 msgr2=0x7fb23406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 0x7fb23406ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb244006fd0 tx=0x7fb244008040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-04-12T19:50:32.512 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 msgr2=0x7fb2540cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009f80 tx=0x7fb24c004ab0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 shutdown_connections 2024-04-12T19:50:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb23406c9c0 0x7fb23406ee80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2540cba20 0x7fb2540cbea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2540cde80 0x7fb2540ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 --2- 172.21.15.32:0/2129743211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2380051c0 0x7fb254158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 >> 172.21.15.32:0/2129743211 conn(0x7fb254073480 msgr2=0x7fb2540b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 shutdown_connections 2024-04-12T19:50:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.511+0000 7fb23fe00640 1 -- 172.21.15.32:0/2129743211 wait complete. 2024-04-12T19:50:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.518+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fefcc06e1a0 con 0x7fefcc06ffe0 2024-04-12T19:50:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.518+0000 7fefc2a00640 1 -- 172.21.15.32:0/1918162088 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefbc05ce70 con 0x7fefcc06ffe0 2024-04-12T19:50:32.519 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 msgr2=0x7fefac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 0x7fefac06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefb8036000 tx=0x7fefb803d040 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 msgr2=0x7fefcc157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c4a0 tx=0x7fefbc00c960 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 shutdown_connections 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fefac06c8f0 0x7fefac06edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefcc15e8d0 0x7fefcc14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefcc0b06b0 0x7fefcc14d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 --2- 172.21.15.32:0/1918162088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefcc06ffe0 0x7fefcc157660 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 >> 172.21.15.32:0/1918162088 conn(0x7fefcc0b6b20 msgr2=0x7fefcc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 shutdown_connections 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7fefd1400640 1 -- 172.21.15.32:0/1918162088 wait complete. 2024-04-12T19:50:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.522+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f795c15a250 con 0x7f795c0b3ca0 2024-04-12T19:50:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.523+0000 7f7958c00640 1 -- 172.21.15.32:0/1212429962 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f794c058e20 con 0x7f795c0b3ca0 2024-04-12T19:50:32.524 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-12T19:50:32.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-04-12T19:50:32.525 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 msgr2=0x7f793c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 0x7f793c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7954036000 tx=0x7f7954009000 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 msgr2=0x7f795c201070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c470 tx=0x7f794c001e70 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 shutdown_connections 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f793c06c8f0 0x7f793c06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f795c15aaa0 0x7f795c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f795c0b3ca0 0x7f795c201070 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c470 tx=0x7f794c001e70 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 --2- 172.21.15.32:0/1212429962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f795c072810 0x7f795c14f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.526+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 >> 172.21.15.32:0/1212429962 conn(0x7f795c0b2460 msgr2=0x7f795c0b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.527+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 shutdown_connections 2024-04-12T19:50:32.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.527+0000 7f7952000640 1 -- 172.21.15.32:0/1212429962 wait complete. 2024-04-12T19:50:32.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-04-12T19:50:32.530 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-04-12T19:50:32.539 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-04-12T19:50:32.544 DEBUG:teuthology.parallel:result is None 2024-04-12T19:50:32.544 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-12T19:50:32.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:50:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.787+0000 7fed2ae00640 1 Processor -- start 2024-04-12T19:50:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.787+0000 7fed2ae00640 1 -- start start 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.788+0000 7fed2ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fed2415b860 0x7fed2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.788+0000 7fed2ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed240b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.788+0000 7fed2ae00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed240cdfb0 con 0x7fed2415b480 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed2ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fed2415f060 con 0x7fed2415b860 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed2ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed2415fec0 con 0x7fed2406f320 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed29a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed2415b480 0x7fed240cd8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43302/0 (socket says 172.21.15.32:43302) 2024-04-12T19:50:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed29a00640 1 -- 172.21.15.32:0/1069981980 learned_addr learned my addr 172.21.15.32:0/1069981980 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158550724 0 0) 0x7fed240cdfb0 con 0x7fed2415b480 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed0c003520 con 0x7fed2415b480 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed29000640 1 --2- 172.21.15.32:0/1069981980 >> v2:172.21.15.169:3300/0 conn(0x7fed2415b860 0x7fed2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.789+0000 7fed2a400640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed240b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 904415845 0 0) 0x7fed0c003520 con 0x7fed2415b480 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 msgr2=0x7fed240b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed240b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 >> v2:172.21.15.169:3300/0 conn(0x7fed2415b860 msgr2=0x7fed2406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 --2- 172.21.15.32:0/1069981980 >> v2:172.21.15.169:3300/0 conn(0x7fed2415b860 0x7fed2406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2415f2b0 con 0x7fed2415b480 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fed180019f0 con 0x7fed2415b480 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 >> v1:172.21.15.32:6789/0 conn(0x7fed2415b480 legacy=0x7fed240cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed0c003fe0 0x7fed0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0c004d40 0x7fed0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed2415f060 con 0x7fed0c004d40 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed2415fec0 con 0x7fed0c003fe0 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c00bac0 con 0x7fed2406f320 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0c004d40 0x7fed0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed29a00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed29000640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed0c003fe0 0x7fed0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 -- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 msgr2=0x7fed0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2406f320 0x7fed0c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 -- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed0c003fe0 msgr2=0x7fed0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed0c003fe0 0x7fed0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.790+0000 7fed2a400640 1 -- 172.21.15.32:0/1069981980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c003520 con 0x7fed0c004d40 2024-04-12T19:50:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.791+0000 7fed2a400640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0c004d40 0x7fed0c0051c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00ba30 tx=0x7fed0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab67833804145332 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c036020 con 0x7fed0c004d40 2024-04-12T19:50:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c032640 con 0x7fed0c004d40 2024-04-12T19:50:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed23e00640 1 -- 172.21.15.32:0/1069981980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00b600 con 0x7fed0c004d40 2024-04-12T19:50:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 -- 172.21.15.32:0/1069981980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0c004d40 msgr2=0x7fed0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 --2- 172.21.15.32:0/1069981980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0c004d40 0x7fed0c0051c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00ba30 tx=0x7fed0c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2a400640 1 -- 172.21.15.32:0/1069981980 reap_dead start 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 -- 172.21.15.32:0/1069981980 shutdown_connections 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 -- 172.21.15.32:0/1069981980 >> 172.21.15.32:0/1069981980 conn(0x7fed2406b930 msgr2=0x7fed2406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 -- 172.21.15.32:0/1069981980 shutdown_connections 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.792+0000 7fed2ae00640 1 -- 172.21.15.32:0/1069981980 wait complete. 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.794+0000 7fed2ae00640 1 Processor -- start 2024-04-12T19:50:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.794+0000 7fed2ae00640 1 -- start start 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed0c004d40 0x7fed2414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed24143450 0x7fed24149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed2415fb50 con 0x7fed24148c40 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed2415f870 con 0x7fed24143450 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed240cd550 con 0x7fed0c004d40 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33042/0 (socket says 172.21.15.32:33042) 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.795+0000 7fed2a400640 1 -- 172.21.15.32:0/2019346957 learned_addr learned my addr 172.21.15.32:0/2019346957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:32.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 -- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed0c004d40 msgr2=0x7fed2414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed29a00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed0c004d40 0x7fed2414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed0c004d40 0x7fed2414b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 -- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed24143450 msgr2=0x7fed24149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed24143450 0x7fed24149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 -- 172.21.15.32:0/2019346957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed241440c0 con 0x7fed24148c40 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2a400640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00bb50 tx=0x7fed1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c041070 con 0x7fed24148c40 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c030950 con 0x7fed24148c40 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c036040 con 0x7fed24148c40 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2415bdc0 con 0x7fed24148c40 2024-04-12T19:50:32.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.796+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2414abf0 con 0x7fed24148c40 2024-04-12T19:50:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.797+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2415c0d0 con 0x7fed24148c40 2024-04-12T19:50:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.798+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed1c04c020 con 0x7fed24148c40 2024-04-12T19:50:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.799+0000 7fed21600640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:32.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.800+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fed1c0ada10 con 0x7fed24148c40 2024-04-12T19:50:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.800+0000 7fed29a00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:32.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.800+0000 7fed29a00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 0x7fed0406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed18002230 tx=0x7fed18002120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:32.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.804+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed241440c0 con 0x7fed24148c40 2024-04-12T19:50:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:32.998+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed240755c0 con 0x7fed0406c8f0 2024-04-12T19:50:33.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.009+0000 7fed21600640 1 -- 172.21.15.32:0/2019346957 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7fed240755c0 con 0x7fed0406c8f0 2024-04-12T19:50:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:33.014 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.015+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 msgr2=0x7fed0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.015+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 0x7fed0406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fed18002230 tx=0x7fed18002120 comp rx=0 tx=0).stop 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 msgr2=0x7fed241490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00bb50 tx=0x7fed1c004730 comp rx=0 tx=0).stop 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 shutdown_connections 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24148c40 0x7fed241490c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed24143450 0x7fed24149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 --2- 172.21.15.32:0/2019346957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed0c004d40 0x7fed2414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 >> 172.21.15.32:0/2019346957 conn(0x7fed2406b930 msgr2=0x7fed2414d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 shutdown_connections 2024-04-12T19:50:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.016+0000 7fed2ae00640 1 -- 172.21.15.32:0/2019346957 wait complete. 2024-04-12T19:50:33.034 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-12T19:50:31.628400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.979038"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689002+0000","last_change":"2024-04-12T19:50:24.687884+0000","last_active":"2024-04-12T19:50:25.689002+0000","last_peered":"2024-04-12T19:50:25.689002+0000","last_clean":"2024-04-12T19:50:25.689002+0000","last_became_active":"2024-04-12T19:50:24.686545+0000","last_became_peered":"2024-04-12T19:50:24.686545+0000","last_unstale":"2024-04-12T19:50:25.689002+0000","last_undegraded":"2024-04-12T19:50:25.689002+0000","last_fullsized":"2024-04-12T19:50:25.689002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194273+0000","last_change":"2024-04-12T19:50:24.693720+0000","last_active":"2024-04-12T19:50:26.194273+0000","last_peered":"2024-04-12T19:50:26.194273+0000","last_clean":"2024-04-12T19:50:26.194273+0000","last_became_active":"2024-04-12T19:50:24.693332+0000","last_became_peered":"2024-04-12T19:50:24.693332+0000","last_unstale":"2024-04-12T19:50:26.194273+0000","last_undegraded":"2024-04-12T19:50:26.194273+0000","last_fullsized":"2024-04-12T19:50:26.194273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194432+0000","last_change":"2024-04-12T19:50:24.688591+0000","last_active":"2024-04-12T19:50:26.194432+0000","last_peered":"2024-04-12T19:50:26.194432+0000","last_clean":"2024-04-12T19:50:26.194432+0000","last_became_active":"2024-04-12T19:50:24.688345+0000","last_became_peered":"2024-04-12T19:50:24.688345+0000","last_unstale":"2024-04-12T19:50:26.194432+0000","last_undegraded":"2024-04-12T19:50:26.194432+0000","last_fullsized":"2024-04-12T19:50:26.194432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675845+0000","last_change":"2024-04-12T19:50:24.680503+0000","last_active":"2024-04-12T19:50:25.675845+0000","last_peered":"2024-04-12T19:50:25.675845+0000","last_clean":"2024-04-12T19:50:25.675845+0000","last_became_active":"2024-04-12T19:50:24.674820+0000","last_became_peered":"2024-04-12T19:50:24.674820+0000","last_unstale":"2024-04-12T19:50:25.675845+0000","last_undegraded":"2024-04-12T19:50:25.675845+0000","last_fullsized":"2024-04-12T19:50:25.675845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688640+0000","last_change":"2024-04-12T19:50:24.688597+0000","last_active":"2024-04-12T19:50:25.688640+0000","last_peered":"2024-04-12T19:50:25.688640+0000","last_clean":"2024-04-12T19:50:25.688640+0000","last_became_active":"2024-04-12T19:50:24.688308+0000","last_became_peered":"2024-04-12T19:50:24.688308+0000","last_unstale":"2024-04-12T19:50:25.688640+0000","last_undegraded":"2024-04-12T19:50:25.688640+0000","last_fullsized":"2024-04-12T19:50:25.688640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675161+0000","last_change":"2024-04-12T19:50:24.680554+0000","last_active":"2024-04-12T19:50:25.675161+0000","last_peered":"2024-04-12T19:50:25.675161+0000","last_clean":"2024-04-12T19:50:25.675161+0000","last_became_active":"2024-04-12T19:50:24.675178+0000","last_became_peered":"2024-04-12T19:50:24.675178+0000","last_unstale":"2024-04-12T19:50:25.675161+0000","last_undegraded":"2024-04-12T19:50:25.675161+0000","last_fullsized":"2024-04-12T19:50:25.675161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688158+0000","last_change":"2024-04-12T19:50:24.677924+0000","last_active":"2024-04-12T19:50:25.688158+0000","last_peered":"2024-04-12T19:50:25.688158+0000","last_clean":"2024-04-12T19:50:25.688158+0000","last_became_active":"2024-04-12T19:50:24.677414+0000","last_became_peered":"2024-04-12T19:50:24.677414+0000","last_unstale":"2024-04-12T19:50:25.688158+0000","last_undegraded":"2024-04-12T19:50:25.688158+0000","last_fullsized":"2024-04-12T19:50:25.688158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193170+0000","last_change":"2024-04-12T19:50:24.690479+0000","last_active":"2024-04-12T19:50:26.193170+0000","last_peered":"2024-04-12T19:50:26.193170+0000","last_clean":"2024-04-12T19:50:26.193170+0000","last_became_active":"2024-04-12T19:50:24.690010+0000","last_became_peered":"2024-04-12T19:50:24.690010+0000","last_unstale":"2024-04-12T19:50:26.193170+0000","last_undegraded":"2024-04-12T19:50:26.193170+0000","last_fullsized":"2024-04-12T19:50:26.193170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673588+0000","last_change":"2024-04-12T19:50:24.679542+0000","last_active":"2024-04-12T19:50:25.673588+0000","last_peered":"2024-04-12T19:50:25.673588+0000","last_clean":"2024-04-12T19:50:25.673588+0000","last_became_active":"2024-04-12T19:50:24.678975+0000","last_became_peered":"2024-04-12T19:50:24.678975+0000","last_unstale":"2024-04-12T19:50:25.673588+0000","last_undegraded":"2024-04-12T19:50:25.673588+0000","last_fullsized":"2024-04-12T19:50:25.673588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688977+0000","last_change":"2024-04-12T19:50:24.681572+0000","last_active":"2024-04-12T19:50:25.688977+0000","last_peered":"2024-04-12T19:50:25.688977+0000","last_clean":"2024-04-12T19:50:25.688977+0000","last_became_active":"2024-04-12T19:50:24.681309+0000","last_became_peered":"2024-04-12T19:50:24.681309+0000","last_unstale":"2024-04-12T19:50:25.688977+0000","last_undegraded":"2024-04-12T19:50:25.688977+0000","last_fullsized":"2024-04-12T19:50:25.688977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688257+0000","last_change":"2024-04-12T19:50:24.678439+0000","last_active":"2024-04-12T19:50:25.688257+0000","last_peered":"2024-04-12T19:50:25.688257+0000","last_clean":"2024-04-12T19:50:25.688257+0000","last_became_active":"2024-04-12T19:50:24.677934+0000","last_became_peered":"2024-04-12T19:50:24.677934+0000","last_unstale":"2024-04-12T19:50:25.688257+0000","last_undegraded":"2024-04-12T19:50:25.688257+0000","last_fullsized":"2024-04-12T19:50:25.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688579+0000","last_change":"2024-04-12T19:50:24.683054+0000","last_active":"2024-04-12T19:50:25.688579+0000","last_peered":"2024-04-12T19:50:25.688579+0000","last_clean":"2024-04-12T19:50:25.688579+0000","last_became_active":"2024-04-12T19:50:24.682749+0000","last_became_peered":"2024-04-12T19:50:24.682749+0000","last_unstale":"2024-04-12T19:50:25.688579+0000","last_undegraded":"2024-04-12T19:50:25.688579+0000","last_fullsized":"2024-04-12T19:50:25.688579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675969+0000","last_change":"2024-04-12T19:50:24.700812+0000","last_active":"2024-04-12T19:50:25.675969+0000","last_peered":"2024-04-12T19:50:25.675969+0000","last_clean":"2024-04-12T19:50:25.675969+0000","last_became_active":"2024-04-12T19:50:24.700470+0000","last_became_peered":"2024-04-12T19:50:24.700470+0000","last_unstale":"2024-04-12T19:50:25.675969+0000","last_undegraded":"2024-04-12T19:50:25.675969+0000","last_fullsized":"2024-04-12T19:50:25.675969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676868+0000","last_change":"2024-04-12T19:50:24.678150+0000","last_active":"2024-04-12T19:50:25.676868+0000","last_peered":"2024-04-12T19:50:25.676868+0000","last_clean":"2024-04-12T19:50:25.676868+0000","last_became_active":"2024-04-12T19:50:24.677811+0000","last_became_peered":"2024-04-12T19:50:24.677811+0000","last_unstale":"2024-04-12T19:50:25.676868+0000","last_undegraded":"2024-04-12T19:50:25.676868+0000","last_fullsized":"2024-04-12T19:50:25.676868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193225+0000","last_change":"2024-04-12T19:50:24.701334+0000","last_active":"2024-04-12T19:50:26.193225+0000","last_peered":"2024-04-12T19:50:26.193225+0000","last_clean":"2024-04-12T19:50:26.193225+0000","last_became_active":"2024-04-12T19:50:24.701046+0000","last_became_peered":"2024-04-12T19:50:24.701046+0000","last_unstale":"2024-04-12T19:50:26.193225+0000","last_undegraded":"2024-04-12T19:50:26.193225+0000","last_fullsized":"2024-04-12T19:50:26.193225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184164+0000","last_change":"2024-04-12T19:50:24.678787+0000","last_active":"2024-04-12T19:50:26.184164+0000","last_peered":"2024-04-12T19:50:26.184164+0000","last_clean":"2024-04-12T19:50:26.184164+0000","last_became_active":"2024-04-12T19:50:24.678537+0000","last_became_peered":"2024-04-12T19:50:24.678537+0000","last_unstale":"2024-04-12T19:50:26.184164+0000","last_undegraded":"2024-04-12T19:50:26.184164+0000","last_fullsized":"2024-04-12T19:50:26.184164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185746+0000","last_change":"2024-04-12T19:50:24.680249+0000","last_active":"2024-04-12T19:50:26.185746+0000","last_peered":"2024-04-12T19:50:26.185746+0000","last_clean":"2024-04-12T19:50:26.185746+0000","last_became_active":"2024-04-12T19:50:24.679835+0000","last_became_peered":"2024-04-12T19:50:24.679835+0000","last_unstale":"2024-04-12T19:50:26.185746+0000","last_undegraded":"2024-04-12T19:50:26.185746+0000","last_fullsized":"2024-04-12T19:50:26.185746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:17:07.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.750013+0000","last_change":"2024-04-12T19:50:24.697806+0000","last_active":"2024-04-12T19:50:25.750013+0000","last_peered":"2024-04-12T19:50:25.750013+0000","last_clean":"2024-04-12T19:50:25.750013+0000","last_became_active":"2024-04-12T19:50:24.697515+0000","last_became_peered":"2024-04-12T19:50:24.697515+0000","last_unstale":"2024-04-12T19:50:25.750013+0000","last_undegraded":"2024-04-12T19:50:25.750013+0000","last_fullsized":"2024-04-12T19:50:25.750013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:12:47.126078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185806+0000","last_change":"2024-04-12T19:50:24.684475+0000","last_active":"2024-04-12T19:50:26.185806+0000","last_peered":"2024-04-12T19:50:26.185806+0000","last_clean":"2024-04-12T19:50:26.185806+0000","last_became_active":"2024-04-12T19:50:24.684052+0000","last_became_peered":"2024-04-12T19:50:24.684052+0000","last_unstale":"2024-04-12T19:50:26.185806+0000","last_undegraded":"2024-04-12T19:50:26.185806+0000","last_fullsized":"2024-04-12T19:50:26.185806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:19:38.175136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688476+0000","last_change":"2024-04-12T19:50:24.679515+0000","last_active":"2024-04-12T19:50:25.688476+0000","last_peered":"2024-04-12T19:50:25.688476+0000","last_clean":"2024-04-12T19:50:25.688476+0000","last_became_active":"2024-04-12T19:50:24.678757+0000","last_became_peered":"2024-04-12T19:50:24.678757+0000","last_unstale":"2024-04-12T19:50:25.688476+0000","last_undegraded":"2024-04-12T19:50:25.688476+0000","last_fullsized":"2024-04-12T19:50:25.688476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682423+0000","last_change":"2024-04-12T19:50:24.674490+0000","last_active":"2024-04-12T19:50:25.682423+0000","last_peered":"2024-04-12T19:50:25.682423+0000","last_clean":"2024-04-12T19:50:25.682423+0000","last_became_active":"2024-04-12T19:50:24.674131+0000","last_became_peered":"2024-04-12T19:50:24.674131+0000","last_unstale":"2024-04-12T19:50:25.682423+0000","last_undegraded":"2024-04-12T19:50:25.682423+0000","last_fullsized":"2024-04-12T19:50:25.682423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673126+0000","last_change":"2024-04-12T19:50:24.673011+0000","last_active":"2024-04-12T19:50:25.673126+0000","last_peered":"2024-04-12T19:50:25.673126+0000","last_clean":"2024-04-12T19:50:25.673126+0000","last_became_active":"2024-04-12T19:50:24.672490+0000","last_became_peered":"2024-04-12T19:50:24.672490+0000","last_unstale":"2024-04-12T19:50:25.673126+0000","last_undegraded":"2024-04-12T19:50:25.673126+0000","last_fullsized":"2024-04-12T19:50:25.673126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193541+0000","last_change":"2024-04-12T19:50:24.692657+0000","last_active":"2024-04-12T19:50:26.193541+0000","last_peered":"2024-04-12T19:50:26.193541+0000","last_clean":"2024-04-12T19:50:26.193541+0000","last_became_active":"2024-04-12T19:50:24.692335+0000","last_became_peered":"2024-04-12T19:50:24.692335+0000","last_unstale":"2024-04-12T19:50:26.193541+0000","last_undegraded":"2024-04-12T19:50:26.193541+0000","last_fullsized":"2024-04-12T19:50:26.193541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184179+0000","last_change":"2024-04-12T19:50:24.692524+0000","last_active":"2024-04-12T19:50:26.184179+0000","last_peered":"2024-04-12T19:50:26.184179+0000","last_clean":"2024-04-12T19:50:26.184179+0000","last_became_active":"2024-04-12T19:50:24.692239+0000","last_became_peered":"2024-04-12T19:50:24.692239+0000","last_unstale":"2024-04-12T19:50:26.184179+0000","last_undegraded":"2024-04-12T19:50:26.184179+0000","last_fullsized":"2024-04-12T19:50:26.184179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193671+0000","last_change":"2024-04-12T19:50:24.686815+0000","last_active":"2024-04-12T19:50:26.193671+0000","last_peered":"2024-04-12T19:50:26.193671+0000","last_clean":"2024-04-12T19:50:26.193671+0000","last_became_active":"2024-04-12T19:50:24.686192+0000","last_became_peered":"2024-04-12T19:50:24.686192+0000","last_unstale":"2024-04-12T19:50:26.193671+0000","last_undegraded":"2024-04-12T19:50:26.193671+0000","last_fullsized":"2024-04-12T19:50:26.193671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676094+0000","last_change":"2024-04-12T19:50:24.686489+0000","last_active":"2024-04-12T19:50:25.676094+0000","last_peered":"2024-04-12T19:50:25.676094+0000","last_clean":"2024-04-12T19:50:25.676094+0000","last_became_active":"2024-04-12T19:50:24.685753+0000","last_became_peered":"2024-04-12T19:50:24.685753+0000","last_unstale":"2024-04-12T19:50:25.676094+0000","last_undegraded":"2024-04-12T19:50:25.676094+0000","last_fullsized":"2024-04-12T19:50:25.676094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676600+0000","last_change":"2024-04-12T19:50:24.675093+0000","last_active":"2024-04-12T19:50:25.676600+0000","last_peered":"2024-04-12T19:50:25.676600+0000","last_clean":"2024-04-12T19:50:25.676600+0000","last_became_active":"2024-04-12T19:50:24.674798+0000","last_became_peered":"2024-04-12T19:50:24.674798+0000","last_unstale":"2024-04-12T19:50:25.676600+0000","last_undegraded":"2024-04-12T19:50:25.676600+0000","last_fullsized":"2024-04-12T19:50:25.676600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676605+0000","last_change":"2024-04-12T19:50:24.681961+0000","last_active":"2024-04-12T19:50:25.676605+0000","last_peered":"2024-04-12T19:50:25.676605+0000","last_clean":"2024-04-12T19:50:25.676605+0000","last_became_active":"2024-04-12T19:50:24.681596+0000","last_became_peered":"2024-04-12T19:50:24.681596+0000","last_unstale":"2024-04-12T19:50:25.676605+0000","last_undegraded":"2024-04-12T19:50:25.676605+0000","last_fullsized":"2024-04-12T19:50:25.676605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194168+0000","last_change":"2024-04-12T19:50:24.688114+0000","last_active":"2024-04-12T19:50:26.194168+0000","last_peered":"2024-04-12T19:50:26.194168+0000","last_clean":"2024-04-12T19:50:26.194168+0000","last_became_active":"2024-04-12T19:50:24.687826+0000","last_became_peered":"2024-04-12T19:50:24.687826+0000","last_unstale":"2024-04-12T19:50:26.194168+0000","last_undegraded":"2024-04-12T19:50:26.194168+0000","last_fullsized":"2024-04-12T19:50:26.194168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184379+0000","last_change":"2024-04-12T19:50:24.697023+0000","last_active":"2024-04-12T19:50:26.184379+0000","last_peered":"2024-04-12T19:50:26.184379+0000","last_clean":"2024-04-12T19:50:26.184379+0000","last_became_active":"2024-04-12T19:50:24.696739+0000","last_became_peered":"2024-04-12T19:50:24.696739+0000","last_unstale":"2024-04-12T19:50:26.184379+0000","last_undegraded":"2024-04-12T19:50:26.184379+0000","last_fullsized":"2024-04-12T19:50:26.184379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:46:45.869946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749594+0000","last_change":"2024-04-12T19:50:24.675081+0000","last_active":"2024-04-12T19:50:25.749594+0000","last_peered":"2024-04-12T19:50:25.749594+0000","last_clean":"2024-04-12T19:50:25.749594+0000","last_became_active":"2024-04-12T19:50:24.674812+0000","last_became_peered":"2024-04-12T19:50:24.674812+0000","last_unstale":"2024-04-12T19:50:25.749594+0000","last_undegraded":"2024-04-12T19:50:25.749594+0000","last_fullsized":"2024-04-12T19:50:25.749594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:51:04.796842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.674913+0000","last_change":"2024-04-12T19:50:24.687854+0000","last_active":"2024-04-12T19:50:25.674913+0000","last_peered":"2024-04-12T19:50:25.674913+0000","last_clean":"2024-04-12T19:50:25.674913+0000","last_became_active":"2024-04-12T19:50:24.685605+0000","last_became_peered":"2024-04-12T19:50:24.685605+0000","last_unstale":"2024-04-12T19:50:25.674913+0000","last_undegraded":"2024-04-12T19:50:25.674913+0000","last_fullsized":"2024-04-12T19:50:25.674913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:11.962955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673270+0000","last_change":"2024-04-12T19:50:24.694100+0000","last_active":"2024-04-12T19:50:25.673270+0000","last_peered":"2024-04-12T19:50:25.673270+0000","last_clean":"2024-04-12T19:50:25.673270+0000","last_became_active":"2024-04-12T19:50:24.693828+0000","last_became_peered":"2024-04-12T19:50:24.693828+0000","last_unstale":"2024-04-12T19:50:25.673270+0000","last_undegraded":"2024-04-12T19:50:25.673270+0000","last_fullsized":"2024-04-12T19:50:25.673270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194122+0000","last_change":"2024-04-12T19:50:23.685737+0000","last_active":"2024-04-12T19:50:26.194122+0000","last_peered":"2024-04-12T19:50:26.194122+0000","last_clean":"2024-04-12T19:50:26.194122+0000","last_became_active":"2024-04-12T19:50:23.685371+0000","last_became_peered":"2024-04-12T19:50:23.685371+0000","last_unstale":"2024-04-12T19:50:26.194122+0000","last_undegraded":"2024-04-12T19:50:26.194122+0000","last_fullsized":"2024-04-12T19:50:26.194122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682544+0000","last_change":"2024-04-12T19:50:24.681008+0000","last_active":"2024-04-12T19:50:25.682544+0000","last_peered":"2024-04-12T19:50:25.682544+0000","last_clean":"2024-04-12T19:50:25.682544+0000","last_became_active":"2024-04-12T19:50:24.680689+0000","last_became_peered":"2024-04-12T19:50:24.680689+0000","last_unstale":"2024-04-12T19:50:25.682544+0000","last_undegraded":"2024-04-12T19:50:25.682544+0000","last_fullsized":"2024-04-12T19:50:25.682544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184687+0000","last_change":"2024-04-12T19:50:23.691933+0000","last_active":"2024-04-12T19:50:26.184687+0000","last_peered":"2024-04-12T19:50:26.184687+0000","last_clean":"2024-04-12T19:50:26.184687+0000","last_became_active":"2024-04-12T19:50:23.691622+0000","last_became_peered":"2024-04-12T19:50:23.691622+0000","last_unstale":"2024-04-12T19:50:26.184687+0000","last_undegraded":"2024-04-12T19:50:26.184687+0000","last_fullsized":"2024-04-12T19:50:26.184687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675732+0000","last_change":"2024-04-12T19:50:23.691962+0000","last_active":"2024-04-12T19:50:25.675732+0000","last_peered":"2024-04-12T19:50:25.675732+0000","last_clean":"2024-04-12T19:50:25.675732+0000","last_became_active":"2024-04-12T19:50:23.691571+0000","last_became_peered":"2024-04-12T19:50:23.691571+0000","last_unstale":"2024-04-12T19:50:25.675732+0000","last_undegraded":"2024-04-12T19:50:25.675732+0000","last_fullsized":"2024-04-12T19:50:25.675732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:10:39.564249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193629+0000","last_change":"2024-04-12T19:50:24.690993+0000","last_active":"2024-04-12T19:50:26.193629+0000","last_peered":"2024-04-12T19:50:26.193629+0000","last_clean":"2024-04-12T19:50:26.193629+0000","last_became_active":"2024-04-12T19:50:24.690721+0000","last_became_peered":"2024-04-12T19:50:24.690721+0000","last_unstale":"2024-04-12T19:50:26.193629+0000","last_undegraded":"2024-04-12T19:50:26.193629+0000","last_fullsized":"2024-04-12T19:50:26.193629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184883+0000","last_change":"2024-04-12T19:50:23.689397+0000","last_active":"2024-04-12T19:50:26.184883+0000","last_peered":"2024-04-12T19:50:26.184883+0000","last_clean":"2024-04-12T19:50:26.184883+0000","last_became_active":"2024-04-12T19:50:23.688493+0000","last_became_peered":"2024-04-12T19:50:23.688493+0000","last_unstale":"2024-04-12T19:50:26.184883+0000","last_undegraded":"2024-04-12T19:50:26.184883+0000","last_fullsized":"2024-04-12T19:50:26.184883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682299+0000","last_change":"2024-04-12T19:50:24.667019+0000","last_active":"2024-04-12T19:50:25.682299+0000","last_peered":"2024-04-12T19:50:25.682299+0000","last_clean":"2024-04-12T19:50:25.682299+0000","last_became_active":"2024-04-12T19:50:24.666763+0000","last_became_peered":"2024-04-12T19:50:24.666763+0000","last_unstale":"2024-04-12T19:50:25.682299+0000","last_undegraded":"2024-04-12T19:50:25.682299+0000","last_fullsized":"2024-04-12T19:50:25.682299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673739+0000","last_change":"2024-04-12T19:50:23.686496+0000","last_active":"2024-04-12T19:50:25.673739+0000","last_peered":"2024-04-12T19:50:25.673739+0000","last_clean":"2024-04-12T19:50:25.673739+0000","last_became_active":"2024-04-12T19:50:23.684913+0000","last_became_peered":"2024-04-12T19:50:23.684913+0000","last_unstale":"2024-04-12T19:50:25.673739+0000","last_undegraded":"2024-04-12T19:50:25.673739+0000","last_fullsized":"2024-04-12T19:50:25.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676067+0000","last_change":"2024-04-12T19:50:24.693409+0000","last_active":"2024-04-12T19:50:25.676067+0000","last_peered":"2024-04-12T19:50:25.676067+0000","last_clean":"2024-04-12T19:50:25.676067+0000","last_became_active":"2024-04-12T19:50:24.686066+0000","last_became_peered":"2024-04-12T19:50:24.686066+0000","last_unstale":"2024-04-12T19:50:25.676067+0000","last_undegraded":"2024-04-12T19:50:25.676067+0000","last_fullsized":"2024-04-12T19:50:25.676067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689282+0000","last_change":"2024-04-12T19:50:23.685660+0000","last_active":"2024-04-12T19:50:25.689282+0000","last_peered":"2024-04-12T19:50:25.689282+0000","last_clean":"2024-04-12T19:50:25.689282+0000","last_became_active":"2024-04-12T19:50:23.685112+0000","last_became_peered":"2024-04-12T19:50:23.685112+0000","last_unstale":"2024-04-12T19:50:25.689282+0000","last_undegraded":"2024-04-12T19:50:25.689282+0000","last_fullsized":"2024-04-12T19:50:25.689282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194160+0000","last_change":"2024-04-12T19:50:24.687686+0000","last_active":"2024-04-12T19:50:26.194160+0000","last_peered":"2024-04-12T19:50:26.194160+0000","last_clean":"2024-04-12T19:50:26.194160+0000","last_became_active":"2024-04-12T19:50:24.686557+0000","last_became_peered":"2024-04-12T19:50:24.686557+0000","last_unstale":"2024-04-12T19:50:26.194160+0000","last_undegraded":"2024-04-12T19:50:26.194160+0000","last_fullsized":"2024-04-12T19:50:26.194160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749850+0000","last_change":"2024-04-12T19:50:23.688709+0000","last_active":"2024-04-12T19:50:25.749850+0000","last_peered":"2024-04-12T19:50:25.749850+0000","last_clean":"2024-04-12T19:50:25.749850+0000","last_became_active":"2024-04-12T19:50:23.687448+0000","last_became_peered":"2024-04-12T19:50:23.687448+0000","last_unstale":"2024-04-12T19:50:25.749850+0000","last_undegraded":"2024-04-12T19:50:25.749850+0000","last_fullsized":"2024-04-12T19:50:25.749850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:41.154672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184609+0000","last_change":"2024-04-12T19:50:24.692484+0000","last_active":"2024-04-12T19:50:26.184609+0000","last_peered":"2024-04-12T19:50:26.184609+0000","last_clean":"2024-04-12T19:50:26.184609+0000","last_became_active":"2024-04-12T19:50:24.692157+0000","last_became_peered":"2024-04-12T19:50:24.692157+0000","last_unstale":"2024-04-12T19:50:26.184609+0000","last_undegraded":"2024-04-12T19:50:26.184609+0000","last_fullsized":"2024-04-12T19:50:26.184609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:51:48.663863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675846+0000","last_change":"2024-04-12T19:50:23.686058+0000","last_active":"2024-04-12T19:50:25.675846+0000","last_peered":"2024-04-12T19:50:25.675846+0000","last_clean":"2024-04-12T19:50:25.675846+0000","last_became_active":"2024-04-12T19:50:23.685765+0000","last_became_peered":"2024-04-12T19:50:23.685765+0000","last_unstale":"2024-04-12T19:50:25.675846+0000","last_undegraded":"2024-04-12T19:50:25.675846+0000","last_fullsized":"2024-04-12T19:50:25.675846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:04:17.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688457+0000","last_change":"2024-04-12T19:50:24.682664+0000","last_active":"2024-04-12T19:50:25.688457+0000","last_peered":"2024-04-12T19:50:25.688457+0000","last_clean":"2024-04-12T19:50:25.688457+0000","last_became_active":"2024-04-12T19:50:24.682385+0000","last_became_peered":"2024-04-12T19:50:24.682385+0000","last_unstale":"2024-04-12T19:50:25.688457+0000","last_undegraded":"2024-04-12T19:50:25.688457+0000","last_fullsized":"2024-04-12T19:50:25.688457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749812+0000","last_change":"2024-04-12T19:50:23.689886+0000","last_active":"2024-04-12T19:50:25.749812+0000","last_peered":"2024-04-12T19:50:25.749812+0000","last_clean":"2024-04-12T19:50:25.749812+0000","last_became_active":"2024-04-12T19:50:23.689536+0000","last_became_peered":"2024-04-12T19:50:23.689536+0000","last_unstale":"2024-04-12T19:50:25.749812+0000","last_undegraded":"2024-04-12T19:50:25.749812+0000","last_fullsized":"2024-04-12T19:50:25.749812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:30.521133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675832+0000","last_change":"2024-04-12T19:50:24.689487+0000","last_active":"2024-04-12T19:50:25.675832+0000","last_peered":"2024-04-12T19:50:25.675832+0000","last_clean":"2024-04-12T19:50:25.675832+0000","last_became_active":"2024-04-12T19:50:24.687365+0000","last_became_peered":"2024-04-12T19:50:24.687365+0000","last_unstale":"2024-04-12T19:50:25.675832+0000","last_undegraded":"2024-04-12T19:50:25.675832+0000","last_fullsized":"2024-04-12T19:50:25.675832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184932+0000","last_change":"2024-04-12T19:50:23.682121+0000","last_active":"2024-04-12T19:50:26.184932+0000","last_peered":"2024-04-12T19:50:26.184932+0000","last_clean":"2024-04-12T19:50:26.184932+0000","last_became_active":"2024-04-12T19:50:23.681803+0000","last_became_peered":"2024-04-12T19:50:23.681803+0000","last_unstale":"2024-04-12T19:50:26.184932+0000","last_undegraded":"2024-04-12T19:50:26.184932+0000","last_fullsized":"2024-04-12T19:50:26.184932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184717+0000","last_change":"2024-04-12T19:50:24.685212+0000","last_active":"2024-04-12T19:50:26.184717+0000","last_peered":"2024-04-12T19:50:26.184717+0000","last_clean":"2024-04-12T19:50:26.184717+0000","last_became_active":"2024-04-12T19:50:24.684863+0000","last_became_peered":"2024-04-12T19:50:24.684863+0000","last_unstale":"2024-04-12T19:50:26.184717+0000","last_undegraded":"2024-04-12T19:50:26.184717+0000","last_fullsized":"2024-04-12T19:50:26.184717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:36:27.562288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687944+0000","last_change":"2024-04-12T19:50:23.673596+0000","last_active":"2024-04-12T19:50:25.687944+0000","last_peered":"2024-04-12T19:50:25.687944+0000","last_clean":"2024-04-12T19:50:25.687944+0000","last_became_active":"2024-04-12T19:50:23.672686+0000","last_became_peered":"2024-04-12T19:50:23.672686+0000","last_unstale":"2024-04-12T19:50:25.687944+0000","last_undegraded":"2024-04-12T19:50:25.687944+0000","last_fullsized":"2024-04-12T19:50:25.687944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672719+0000","last_change":"2024-04-12T19:50:24.693699+0000","last_active":"2024-04-12T19:50:25.672719+0000","last_peered":"2024-04-12T19:50:25.672719+0000","last_clean":"2024-04-12T19:50:25.672719+0000","last_became_active":"2024-04-12T19:50:24.693376+0000","last_became_peered":"2024-04-12T19:50:24.693376+0000","last_unstale":"2024-04-12T19:50:25.672719+0000","last_undegraded":"2024-04-12T19:50:25.672719+0000","last_fullsized":"2024-04-12T19:50:25.672719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682161+0000","last_change":"2024-04-12T19:50:23.682371+0000","last_active":"2024-04-12T19:50:25.682161+0000","last_peered":"2024-04-12T19:50:25.682161+0000","last_clean":"2024-04-12T19:50:25.682161+0000","last_became_active":"2024-04-12T19:50:23.681683+0000","last_became_peered":"2024-04-12T19:50:23.681683+0000","last_unstale":"2024-04-12T19:50:25.682161+0000","last_undegraded":"2024-04-12T19:50:25.682161+0000","last_fullsized":"2024-04-12T19:50:25.682161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184368+0000","last_change":"2024-04-12T19:50:24.673851+0000","last_active":"2024-04-12T19:50:26.184368+0000","last_peered":"2024-04-12T19:50:26.184368+0000","last_clean":"2024-04-12T19:50:26.184368+0000","last_became_active":"2024-04-12T19:50:24.673551+0000","last_became_peered":"2024-04-12T19:50:24.673551+0000","last_unstale":"2024-04-12T19:50:26.184368+0000","last_undegraded":"2024-04-12T19:50:26.184368+0000","last_fullsized":"2024-04-12T19:50:26.184368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:29:34.148711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689481+0000","last_change":"2024-04-12T19:50:23.691929+0000","last_active":"2024-04-12T19:50:25.689481+0000","last_peered":"2024-04-12T19:50:25.689481+0000","last_clean":"2024-04-12T19:50:25.689481+0000","last_became_active":"2024-04-12T19:50:23.684811+0000","last_became_peered":"2024-04-12T19:50:23.684811+0000","last_unstale":"2024-04-12T19:50:25.689481+0000","last_undegraded":"2024-04-12T19:50:25.689481+0000","last_fullsized":"2024-04-12T19:50:25.689481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676370+0000","last_change":"2024-04-12T19:50:24.691614+0000","last_active":"2024-04-12T19:50:25.676370+0000","last_peered":"2024-04-12T19:50:25.676370+0000","last_clean":"2024-04-12T19:50:25.676370+0000","last_became_active":"2024-04-12T19:50:24.686194+0000","last_became_peered":"2024-04-12T19:50:24.686194+0000","last_unstale":"2024-04-12T19:50:25.676370+0000","last_undegraded":"2024-04-12T19:50:25.676370+0000","last_fullsized":"2024-04-12T19:50:25.676370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683033+0000","last_change":"2024-04-12T19:50:23.685902+0000","last_active":"2024-04-12T19:50:25.683033+0000","last_peered":"2024-04-12T19:50:25.683033+0000","last_clean":"2024-04-12T19:50:25.683033+0000","last_became_active":"2024-04-12T19:50:23.685614+0000","last_became_peered":"2024-04-12T19:50:23.685614+0000","last_unstale":"2024-04-12T19:50:25.683033+0000","last_undegraded":"2024-04-12T19:50:25.683033+0000","last_fullsized":"2024-04-12T19:50:25.683033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689356+0000","last_change":"2024-04-12T19:50:24.674299+0000","last_active":"2024-04-12T19:50:25.689356+0000","last_peered":"2024-04-12T19:50:25.689356+0000","last_clean":"2024-04-12T19:50:25.689356+0000","last_became_active":"2024-04-12T19:50:24.672616+0000","last_became_peered":"2024-04-12T19:50:24.672616+0000","last_unstale":"2024-04-12T19:50:25.689356+0000","last_undegraded":"2024-04-12T19:50:25.689356+0000","last_fullsized":"2024-04-12T19:50:25.689356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194553+0000","last_change":"2024-04-12T19:50:23.685829+0000","last_active":"2024-04-12T19:50:26.194553+0000","last_peered":"2024-04-12T19:50:26.194553+0000","last_clean":"2024-04-12T19:50:26.194553+0000","last_became_active":"2024-04-12T19:50:23.684944+0000","last_became_peered":"2024-04-12T19:50:23.684944+0000","last_unstale":"2024-04-12T19:50:26.194553+0000","last_undegraded":"2024-04-12T19:50:26.194553+0000","last_fullsized":"2024-04-12T19:50:26.194553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688739+0000","last_change":"2024-04-12T19:50:24.680192+0000","last_active":"2024-04-12T19:50:25.688739+0000","last_peered":"2024-04-12T19:50:25.688739+0000","last_clean":"2024-04-12T19:50:25.688739+0000","last_became_active":"2024-04-12T19:50:24.679916+0000","last_became_peered":"2024-04-12T19:50:24.679916+0000","last_unstale":"2024-04-12T19:50:25.688739+0000","last_undegraded":"2024-04-12T19:50:25.688739+0000","last_fullsized":"2024-04-12T19:50:25.688739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676558+0000","last_change":"2024-04-12T19:50:23.693367+0000","last_active":"2024-04-12T19:50:25.676558+0000","last_peered":"2024-04-12T19:50:25.676558+0000","last_clean":"2024-04-12T19:50:25.676558+0000","last_became_active":"2024-04-12T19:50:23.693036+0000","last_became_peered":"2024-04-12T19:50:23.693036+0000","last_unstale":"2024-04-12T19:50:25.676558+0000","last_undegraded":"2024-04-12T19:50:25.676558+0000","last_fullsized":"2024-04-12T19:50:25.676558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185470+0000","last_change":"2024-04-12T19:50:24.674923+0000","last_active":"2024-04-12T19:50:26.185470+0000","last_peered":"2024-04-12T19:50:26.185470+0000","last_clean":"2024-04-12T19:50:26.185470+0000","last_became_active":"2024-04-12T19:50:24.674524+0000","last_became_peered":"2024-04-12T19:50:24.674524+0000","last_unstale":"2024-04-12T19:50:26.185470+0000","last_undegraded":"2024-04-12T19:50:26.185470+0000","last_fullsized":"2024-04-12T19:50:26.185470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:56.439740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672856+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:50:25.672856+0000","last_peered":"2024-04-12T19:50:25.672856+0000","last_clean":"2024-04-12T19:50:25.672856+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:50:25.672856+0000","last_undegraded":"2024-04-12T19:50:25.672856+0000","last_fullsized":"2024-04-12T19:50:25.672856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184860+0000","last_change":"2024-04-12T19:50:23.676923+0000","last_active":"2024-04-12T19:50:26.184860+0000","last_peered":"2024-04-12T19:50:26.184860+0000","last_clean":"2024-04-12T19:50:26.184860+0000","last_became_active":"2024-04-12T19:50:23.676667+0000","last_became_peered":"2024-04-12T19:50:23.676667+0000","last_unstale":"2024-04-12T19:50:26.184860+0000","last_undegraded":"2024-04-12T19:50:26.184860+0000","last_fullsized":"2024-04-12T19:50:26.184860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:29.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682938+0000","last_change":"2024-04-12T19:50:24.673694+0000","last_active":"2024-04-12T19:50:25.682938+0000","last_peered":"2024-04-12T19:50:25.682938+0000","last_clean":"2024-04-12T19:50:25.682938+0000","last_became_active":"2024-04-12T19:50:24.673340+0000","last_became_peered":"2024-04-12T19:50:24.673340+0000","last_unstale":"2024-04-12T19:50:25.682938+0000","last_undegraded":"2024-04-12T19:50:25.682938+0000","last_fullsized":"2024-04-12T19:50:25.682938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184624+0000","last_change":"2024-04-12T19:50:23.690110+0000","last_active":"2024-04-12T19:50:26.184624+0000","last_peered":"2024-04-12T19:50:26.184624+0000","last_clean":"2024-04-12T19:50:26.184624+0000","last_became_active":"2024-04-12T19:50:23.689677+0000","last_became_peered":"2024-04-12T19:50:23.689677+0000","last_unstale":"2024-04-12T19:50:26.184624+0000","last_undegraded":"2024-04-12T19:50:26.184624+0000","last_fullsized":"2024-04-12T19:50:26.184624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:02.364998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676043+0000","last_change":"2024-04-12T19:50:24.675131+0000","last_active":"2024-04-12T19:50:25.676043+0000","last_peered":"2024-04-12T19:50:25.676043+0000","last_clean":"2024-04-12T19:50:25.676043+0000","last_became_active":"2024-04-12T19:50:24.674486+0000","last_became_peered":"2024-04-12T19:50:24.674486+0000","last_unstale":"2024-04-12T19:50:25.676043+0000","last_undegraded":"2024-04-12T19:50:25.676043+0000","last_fullsized":"2024-04-12T19:50:25.676043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:40.879093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749308+0000","last_change":"2024-04-12T19:50:23.690325+0000","last_active":"2024-04-12T19:50:25.749308+0000","last_peered":"2024-04-12T19:50:25.749308+0000","last_clean":"2024-04-12T19:50:25.749308+0000","last_became_active":"2024-04-12T19:50:23.688951+0000","last_became_peered":"2024-04-12T19:50:23.688951+0000","last_unstale":"2024-04-12T19:50:25.749308+0000","last_undegraded":"2024-04-12T19:50:25.749308+0000","last_fullsized":"2024-04-12T19:50:25.749308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:49:41.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193423+0000","last_change":"2024-04-12T19:50:24.698245+0000","last_active":"2024-04-12T19:50:26.193423+0000","last_peered":"2024-04-12T19:50:26.193423+0000","last_clean":"2024-04-12T19:50:26.193423+0000","last_became_active":"2024-04-12T19:50:24.697937+0000","last_became_peered":"2024-04-12T19:50:24.697937+0000","last_unstale":"2024-04-12T19:50:26.193423+0000","last_undegraded":"2024-04-12T19:50:26.193423+0000","last_fullsized":"2024-04-12T19:50:26.193423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185039+0000","last_change":"2024-04-12T19:50:23.689007+0000","last_active":"2024-04-12T19:50:26.185039+0000","last_peered":"2024-04-12T19:50:26.185039+0000","last_clean":"2024-04-12T19:50:26.185039+0000","last_became_active":"2024-04-12T19:50:23.688654+0000","last_became_peered":"2024-04-12T19:50:23.688654+0000","last_unstale":"2024-04-12T19:50:26.185039+0000","last_undegraded":"2024-04-12T19:50:26.185039+0000","last_fullsized":"2024-04-12T19:50:26.185039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:24.585940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688821+0000","last_change":"2024-04-12T19:50:24.680562+0000","last_active":"2024-04-12T19:50:25.688821+0000","last_peered":"2024-04-12T19:50:25.688821+0000","last_clean":"2024-04-12T19:50:25.688821+0000","last_became_active":"2024-04-12T19:50:24.680276+0000","last_became_peered":"2024-04-12T19:50:24.680276+0000","last_unstale":"2024-04-12T19:50:25.688821+0000","last_undegraded":"2024-04-12T19:50:25.688821+0000","last_fullsized":"2024-04-12T19:50:25.688821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193774+0000","last_change":"2024-04-12T19:50:23.685407+0000","last_active":"2024-04-12T19:50:26.193774+0000","last_peered":"2024-04-12T19:50:26.193774+0000","last_clean":"2024-04-12T19:50:26.193774+0000","last_became_active":"2024-04-12T19:50:23.684661+0000","last_became_peered":"2024-04-12T19:50:23.684661+0000","last_unstale":"2024-04-12T19:50:26.193774+0000","last_undegraded":"2024-04-12T19:50:26.193774+0000","last_fullsized":"2024-04-12T19:50:26.193774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675625+0000","last_change":"2024-04-12T19:50:24.686111+0000","last_active":"2024-04-12T19:50:25.675625+0000","last_peered":"2024-04-12T19:50:25.675625+0000","last_clean":"2024-04-12T19:50:25.675625+0000","last_became_active":"2024-04-12T19:50:24.685534+0000","last_became_peered":"2024-04-12T19:50:24.685534+0000","last_unstale":"2024-04-12T19:50:25.675625+0000","last_undegraded":"2024-04-12T19:50:25.675625+0000","last_fullsized":"2024-04-12T19:50:25.675625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185739+0000","last_change":"2024-04-12T19:50:23.676900+0000","last_active":"2024-04-12T19:50:26.185739+0000","last_peered":"2024-04-12T19:50:26.185739+0000","last_clean":"2024-04-12T19:50:26.185739+0000","last_became_active":"2024-04-12T19:50:23.676612+0000","last_became_peered":"2024-04-12T19:50:23.676612+0000","last_unstale":"2024-04-12T19:50:26.185739+0000","last_undegraded":"2024-04-12T19:50:26.185739+0000","last_fullsized":"2024-04-12T19:50:26.185739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:39.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676360+0000","last_change":"2024-04-12T19:50:24.695675+0000","last_active":"2024-04-12T19:50:25.676360+0000","last_peered":"2024-04-12T19:50:25.676360+0000","last_clean":"2024-04-12T19:50:25.676360+0000","last_became_active":"2024-04-12T19:50:24.695245+0000","last_became_peered":"2024-04-12T19:50:24.695245+0000","last_unstale":"2024-04-12T19:50:25.676360+0000","last_undegraded":"2024-04-12T19:50:25.676360+0000","last_fullsized":"2024-04-12T19:50:25.676360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.686136+0000","last_change":"2024-04-12T19:50:23.681572+0000","last_active":"2024-04-12T19:50:25.686136+0000","last_peered":"2024-04-12T19:50:25.686136+0000","last_clean":"2024-04-12T19:50:25.686136+0000","last_became_active":"2024-04-12T19:50:23.679612+0000","last_became_peered":"2024-04-12T19:50:23.679612+0000","last_unstale":"2024-04-12T19:50:25.686136+0000","last_undegraded":"2024-04-12T19:50:25.686136+0000","last_fullsized":"2024-04-12T19:50:25.686136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682950+0000","last_change":"2024-04-12T19:50:24.686596+0000","last_active":"2024-04-12T19:50:25.682950+0000","last_peered":"2024-04-12T19:50:25.682950+0000","last_clean":"2024-04-12T19:50:25.682950+0000","last_became_active":"2024-04-12T19:50:24.685023+0000","last_became_peered":"2024-04-12T19:50:24.685023+0000","last_unstale":"2024-04-12T19:50:25.682950+0000","last_undegraded":"2024-04-12T19:50:25.682950+0000","last_fullsized":"2024-04-12T19:50:25.682950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185887+0000","last_change":"2024-04-12T19:50:23.684680+0000","last_active":"2024-04-12T19:50:26.185887+0000","last_peered":"2024-04-12T19:50:26.185887+0000","last_clean":"2024-04-12T19:50:26.185887+0000","last_became_active":"2024-04-12T19:50:23.684298+0000","last_became_peered":"2024-04-12T19:50:23.684298+0000","last_unstale":"2024-04-12T19:50:26.185887+0000","last_undegraded":"2024-04-12T19:50:26.185887+0000","last_fullsized":"2024-04-12T19:50:26.185887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:06.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683078+0000","last_change":"2024-04-12T19:50:24.676462+0000","last_active":"2024-04-12T19:50:25.683078+0000","last_peered":"2024-04-12T19:50:25.683078+0000","last_clean":"2024-04-12T19:50:25.683078+0000","last_became_active":"2024-04-12T19:50:24.676182+0000","last_became_peered":"2024-04-12T19:50:24.676182+0000","last_unstale":"2024-04-12T19:50:25.683078+0000","last_undegraded":"2024-04-12T19:50:25.683078+0000","last_fullsized":"2024-04-12T19:50:25.683078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687692+0000","last_change":"2024-04-12T19:50:23.684797+0000","last_active":"2024-04-12T19:50:25.687692+0000","last_peered":"2024-04-12T19:50:25.687692+0000","last_clean":"2024-04-12T19:50:25.687692+0000","last_became_active":"2024-04-12T19:50:23.678108+0000","last_became_peered":"2024-04-12T19:50:23.678108+0000","last_unstale":"2024-04-12T19:50:25.687692+0000","last_undegraded":"2024-04-12T19:50:25.687692+0000","last_fullsized":"2024-04-12T19:50:25.687692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184277+0000","last_change":"2024-04-12T19:50:24.681150+0000","last_active":"2024-04-12T19:50:26.184277+0000","last_peered":"2024-04-12T19:50:26.184277+0000","last_clean":"2024-04-12T19:50:26.184277+0000","last_became_active":"2024-04-12T19:50:24.680857+0000","last_became_peered":"2024-04-12T19:50:24.680857+0000","last_unstale":"2024-04-12T19:50:26.184277+0000","last_undegraded":"2024-04-12T19:50:26.184277+0000","last_fullsized":"2024-04-12T19:50:26.184277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682575+0000","last_change":"2024-04-12T19:50:23.691210+0000","last_active":"2024-04-12T19:50:25.682575+0000","last_peered":"2024-04-12T19:50:25.682575+0000","last_clean":"2024-04-12T19:50:25.682575+0000","last_became_active":"2024-04-12T19:50:23.690943+0000","last_became_peered":"2024-04-12T19:50:23.690943+0000","last_unstale":"2024-04-12T19:50:25.682575+0000","last_undegraded":"2024-04-12T19:50:25.682575+0000","last_fullsized":"2024-04-12T19:50:25.682575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682186+0000","last_change":"2024-04-12T19:50:24.667678+0000","last_active":"2024-04-12T19:50:25.682186+0000","last_peered":"2024-04-12T19:50:25.682186+0000","last_clean":"2024-04-12T19:50:25.682186+0000","last_became_active":"2024-04-12T19:50:24.667407+0000","last_became_peered":"2024-04-12T19:50:24.667407+0000","last_unstale":"2024-04-12T19:50:25.682186+0000","last_undegraded":"2024-04-12T19:50:25.682186+0000","last_fullsized":"2024-04-12T19:50:25.682186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676272+0000","last_change":"2024-04-12T19:50:23.690317+0000","last_active":"2024-04-12T19:50:25.676272+0000","last_peered":"2024-04-12T19:50:25.676272+0000","last_clean":"2024-04-12T19:50:25.676272+0000","last_became_active":"2024-04-12T19:50:23.689037+0000","last_became_peered":"2024-04-12T19:50:23.689037+0000","last_unstale":"2024-04-12T19:50:25.676272+0000","last_undegraded":"2024-04-12T19:50:25.676272+0000","last_fullsized":"2024-04-12T19:50:25.676272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749840+0000","last_change":"2024-04-12T19:50:24.681363+0000","last_active":"2024-04-12T19:50:25.749840+0000","last_peered":"2024-04-12T19:50:25.749840+0000","last_clean":"2024-04-12T19:50:25.749840+0000","last_became_active":"2024-04-12T19:50:24.681100+0000","last_became_peered":"2024-04-12T19:50:24.681100+0000","last_unstale":"2024-04-12T19:50:25.749840+0000","last_undegraded":"2024-04-12T19:50:25.749840+0000","last_fullsized":"2024-04-12T19:50:25.749840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:04:43.842205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184678+0000","last_change":"2024-04-12T19:50:23.679805+0000","last_active":"2024-04-12T19:50:26.184678+0000","last_peered":"2024-04-12T19:50:26.184678+0000","last_clean":"2024-04-12T19:50:26.184678+0000","last_became_active":"2024-04-12T19:50:23.679501+0000","last_became_peered":"2024-04-12T19:50:23.679501+0000","last_unstale":"2024-04-12T19:50:26.184678+0000","last_undegraded":"2024-04-12T19:50:26.184678+0000","last_fullsized":"2024-04-12T19:50:26.184678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:44.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184641+0000","last_change":"2024-04-12T19:50:24.678712+0000","last_active":"2024-04-12T19:50:26.184641+0000","last_peered":"2024-04-12T19:50:26.184641+0000","last_clean":"2024-04-12T19:50:26.184641+0000","last_became_active":"2024-04-12T19:50:24.677584+0000","last_became_peered":"2024-04-12T19:50:24.677584+0000","last_unstale":"2024-04-12T19:50:26.184641+0000","last_undegraded":"2024-04-12T19:50:26.184641+0000","last_fullsized":"2024-04-12T19:50:26.184641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:27:08.109541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185338+0000","last_change":"2024-04-12T19:50:24.698869+0000","last_active":"2024-04-12T19:50:26.185338+0000","last_peered":"2024-04-12T19:50:26.185338+0000","last_clean":"2024-04-12T19:50:26.185338+0000","last_became_active":"2024-04-12T19:50:24.698605+0000","last_became_peered":"2024-04-12T19:50:24.698605+0000","last_unstale":"2024-04-12T19:50:26.185338+0000","last_undegraded":"2024-04-12T19:50:26.185338+0000","last_fullsized":"2024-04-12T19:50:26.185338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:26:54.420503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688867+0000","last_change":"2024-04-12T19:50:23.678658+0000","last_active":"2024-04-12T19:50:25.688867+0000","last_peered":"2024-04-12T19:50:25.688867+0000","last_clean":"2024-04-12T19:50:25.688867+0000","last_became_active":"2024-04-12T19:50:23.676520+0000","last_became_peered":"2024-04-12T19:50:23.676520+0000","last_unstale":"2024-04-12T19:50:25.688867+0000","last_undegraded":"2024-04-12T19:50:25.688867+0000","last_fullsized":"2024-04-12T19:50:25.688867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673494+0000","last_change":"2024-04-12T19:50:24.684296+0000","last_active":"2024-04-12T19:50:25.673494+0000","last_peered":"2024-04-12T19:50:25.673494+0000","last_clean":"2024-04-12T19:50:25.673494+0000","last_became_active":"2024-04-12T19:50:24.683867+0000","last_became_peered":"2024-04-12T19:50:24.683867+0000","last_unstale":"2024-04-12T19:50:25.673494+0000","last_undegraded":"2024-04-12T19:50:25.673494+0000","last_fullsized":"2024-04-12T19:50:25.673494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688543+0000","last_change":"2024-04-12T19:50:23.680894+0000","last_active":"2024-04-12T19:50:25.688543+0000","last_peered":"2024-04-12T19:50:25.688543+0000","last_clean":"2024-04-12T19:50:25.688543+0000","last_became_active":"2024-04-12T19:50:23.679663+0000","last_became_peered":"2024-04-12T19:50:23.679663+0000","last_unstale":"2024-04-12T19:50:25.688543+0000","last_undegraded":"2024-04-12T19:50:25.688543+0000","last_fullsized":"2024-04-12T19:50:25.688543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194138+0000","last_change":"2024-04-12T19:50:24.695055+0000","last_active":"2024-04-12T19:50:26.194138+0000","last_peered":"2024-04-12T19:50:26.194138+0000","last_clean":"2024-04-12T19:50:26.194138+0000","last_became_active":"2024-04-12T19:50:24.694565+0000","last_became_peered":"2024-04-12T19:50:24.694565+0000","last_unstale":"2024-04-12T19:50:26.194138+0000","last_undegraded":"2024-04-12T19:50:26.194138+0000","last_fullsized":"2024-04-12T19:50:26.194138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194545+0000","last_change":"2024-04-12T19:50:23.684953+0000","last_active":"2024-04-12T19:50:26.194545+0000","last_peered":"2024-04-12T19:50:26.194545+0000","last_clean":"2024-04-12T19:50:26.194545+0000","last_became_active":"2024-04-12T19:50:23.684555+0000","last_became_peered":"2024-04-12T19:50:23.684555+0000","last_unstale":"2024-04-12T19:50:26.194545+0000","last_undegraded":"2024-04-12T19:50:26.194545+0000","last_fullsized":"2024-04-12T19:50:26.194545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194574+0000","last_change":"2024-04-12T19:50:24.701879+0000","last_active":"2024-04-12T19:50:26.194574+0000","last_peered":"2024-04-12T19:50:26.194574+0000","last_clean":"2024-04-12T19:50:26.194574+0000","last_became_active":"2024-04-12T19:50:24.701573+0000","last_became_peered":"2024-04-12T19:50:24.701573+0000","last_unstale":"2024-04-12T19:50:26.194574+0000","last_undegraded":"2024-04-12T19:50:26.194574+0000","last_fullsized":"2024-04-12T19:50:26.194574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683114+0000","last_change":"2024-04-12T19:50:23.673049+0000","last_active":"2024-04-12T19:50:25.683114+0000","last_peered":"2024-04-12T19:50:25.683114+0000","last_clean":"2024-04-12T19:50:25.683114+0000","last_became_active":"2024-04-12T19:50:23.672610+0000","last_became_peered":"2024-04-12T19:50:23.672610+0000","last_unstale":"2024-04-12T19:50:25.683114+0000","last_undegraded":"2024-04-12T19:50:25.683114+0000","last_fullsized":"2024-04-12T19:50:25.683114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:50:33.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:50:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.296+0000 7f509fe00640 1 Processor -- start 2024-04-12T19:50:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.296+0000 7f509fe00640 1 -- start start 2024-04-12T19:50:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.298+0000 7f509fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f50a00cee00 0x7f50a006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.298+0000 7f509fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.298+0000 7f509fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a015dbb0 con 0x7f50a0071f50 2024-04-12T19:50:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.298+0000 7f509fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f50a006f4f0 con 0x7f50a00cee00 2024-04-12T19:50:33.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.298+0000 7f509fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50a00728f0 con 0x7f50a015d2f0 2024-04-12T19:50:33.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.299+0000 7f509f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.299+0000 7f509ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f50a00cee00 0x7f50a006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.299+0000 7f509f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:55142/0 (socket says 172.21.15.32:55142) 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.299+0000 7f509f400640 1 -- 172.21.15.32:0/3184886021 learned_addr learned my addr 172.21.15.32:0/3184886021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.299+0000 7f509d600640 1 -- 172.21.15.32:0/3184886021 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063377674 0 0) 0x7f50a015dbb0 con 0x7f50a0071f50 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.300+0000 7f509d600640 1 -- 172.21.15.32:0/3184886021 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5084003520 con 0x7f50a0071f50 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.300+0000 7f509f400640 1 -- 172.21.15.32:0/3184886021 >> v2:172.21.15.169:3300/0 conn(0x7f50a00cee00 msgr2=0x7f50a006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.300+0000 7f509f400640 1 --2- 172.21.15.32:0/3184886021 >> v2:172.21.15.169:3300/0 conn(0x7f50a00cee00 0x7f50a006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.300+0000 7f509f400640 1 -- 172.21.15.32:0/3184886021 >> v1:172.21.15.169:6789/0 conn(0x7f50a0071f50 legacy=0x7f50a015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.300+0000 7f509f400640 1 -- 172.21.15.32:0/3184886021 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a015d8d0 con 0x7f50a015d2f0 2024-04-12T19:50:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.301+0000 7f509f400640 1 --2- 172.21.15.32:0/3184886021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5098009e50 tx=0x7f509802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def5a247250ef45c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.301+0000 7f509d600640 1 -- 172.21.15.32:0/3184886021 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509802cb00 con 0x7f50a015d2f0 2024-04-12T19:50:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.301+0000 7f509d600640 1 -- 172.21.15.32:0/3184886021 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509802cca0 con 0x7f50a015d2f0 2024-04-12T19:50:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509d600640 1 -- 172.21.15.32:0/3184886021 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098034610 con 0x7f50a015d2f0 2024-04-12T19:50:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 -- 172.21.15.32:0/3184886021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 msgr2=0x7f50a00723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 --2- 172.21.15.32:0/3184886021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5098009e50 tx=0x7f509802bf60 comp rx=0 tx=0).stop 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 -- 172.21.15.32:0/3184886021 shutdown_connections 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 --2- 172.21.15.32:0/3184886021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a00723b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 --2- 172.21.15.32:0/3184886021 >> v2:172.21.15.169:3300/0 conn(0x7f50a00cee00 0x7f50a006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.302+0000 7f509fe00640 1 -- 172.21.15.32:0/3184886021 >> 172.21.15.32:0/3184886021 conn(0x7f50a006b930 msgr2=0x7f50a006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.303+0000 7f509fe00640 1 -- 172.21.15.32:0/3184886021 shutdown_connections 2024-04-12T19:50:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.303+0000 7f509fe00640 1 -- 172.21.15.32:0/3184886021 wait complete. 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.305+0000 7f509fe00640 1 Processor -- start 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.305+0000 7f509fe00640 1 -- start start 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 0x7f50a00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50a006e3a0 con 0x7f50a0071f50 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50a006e0c0 con 0x7f50a00cee00 2024-04-12T19:50:33.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50a015dda0 con 0x7f50a015d2f0 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.306+0000 7f509ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33052/0 (socket says 172.21.15.32:33052) 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 -- 172.21.15.32:0/1017236915 learned_addr learned my addr 172.21.15.32:0/1017236915 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509e000640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 0x7f50a00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 -- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 msgr2=0x7f50a0155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 -- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 msgr2=0x7f50a00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 0x7f50a00b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 -- 172.21.15.32:0/1017236915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a01583a0 con 0x7f50a0071f50 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509e000640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 0x7f50a00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:33.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.307+0000 7f509ea00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9e0 tx=0x7f509000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.308+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090017070 con 0x7f50a0071f50 2024-04-12T19:50:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.308+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509000c9d0 con 0x7f50a0071f50 2024-04-12T19:50:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.308+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a01f8180 con 0x7f50a0071f50 2024-04-12T19:50:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.308+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000cdc0 con 0x7f50a0071f50 2024-04-12T19:50:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.308+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a0158920 con 0x7f50a0071f50 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.309+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a00755c0 con 0x7f50a0071f50 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.310+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f509001a070 con 0x7f50a0071f50 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.311+0000 7f5096a00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.311+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f509008bc50 con 0x7f50a0071f50 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.311+0000 7f509e000640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.312+0000 7f509e000640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 0x7f508006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b1ca0 tx=0x7f508c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.316+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5090091020 con 0x7f50a0071f50 2024-04-12T19:50:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.507+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50a006cf30 con 0x7f508006c8f0 2024-04-12T19:50:33.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.526+0000 7f5096a00640 1 -- 172.21.15.32:0/1017236915 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7f50a006cf30 con 0x7f508006c8f0 2024-04-12T19:50:33.530 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:33.531 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:50:33.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 msgr2=0x7f508006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 0x7f508006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b1ca0 tx=0x7f508c009040 comp rx=0 tx=0).stop 2024-04-12T19:50:33.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 msgr2=0x7f50a00aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9e0 tx=0x7f509000eea0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 shutdown_connections 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f508006c8f0 0x7f508006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50a015d2f0 0x7f50a0155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50a00cee00 0x7f50a00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 --2- 172.21.15.32:0/1017236915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a0071f50 0x7f50a00aec60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 >> 172.21.15.32:0/1017236915 conn(0x7f50a006b930 msgr2=0x7f50a006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 shutdown_connections 2024-04-12T19:50:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.532+0000 7f509fe00640 1 -- 172.21.15.32:0/1017236915 wait complete. 2024-04-12T19:50:33.550 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-12T19:50:31.628400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.979038"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689002+0000","last_change":"2024-04-12T19:50:24.687884+0000","last_active":"2024-04-12T19:50:25.689002+0000","last_peered":"2024-04-12T19:50:25.689002+0000","last_clean":"2024-04-12T19:50:25.689002+0000","last_became_active":"2024-04-12T19:50:24.686545+0000","last_became_peered":"2024-04-12T19:50:24.686545+0000","last_unstale":"2024-04-12T19:50:25.689002+0000","last_undegraded":"2024-04-12T19:50:25.689002+0000","last_fullsized":"2024-04-12T19:50:25.689002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194273+0000","last_change":"2024-04-12T19:50:24.693720+0000","last_active":"2024-04-12T19:50:26.194273+0000","last_peered":"2024-04-12T19:50:26.194273+0000","last_clean":"2024-04-12T19:50:26.194273+0000","last_became_active":"2024-04-12T19:50:24.693332+0000","last_became_peered":"2024-04-12T19:50:24.693332+0000","last_unstale":"2024-04-12T19:50:26.194273+0000","last_undegraded":"2024-04-12T19:50:26.194273+0000","last_fullsized":"2024-04-12T19:50:26.194273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194432+0000","last_change":"2024-04-12T19:50:24.688591+0000","last_active":"2024-04-12T19:50:26.194432+0000","last_peered":"2024-04-12T19:50:26.194432+0000","last_clean":"2024-04-12T19:50:26.194432+0000","last_became_active":"2024-04-12T19:50:24.688345+0000","last_became_peered":"2024-04-12T19:50:24.688345+0000","last_unstale":"2024-04-12T19:50:26.194432+0000","last_undegraded":"2024-04-12T19:50:26.194432+0000","last_fullsized":"2024-04-12T19:50:26.194432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675845+0000","last_change":"2024-04-12T19:50:24.680503+0000","last_active":"2024-04-12T19:50:25.675845+0000","last_peered":"2024-04-12T19:50:25.675845+0000","last_clean":"2024-04-12T19:50:25.675845+0000","last_became_active":"2024-04-12T19:50:24.674820+0000","last_became_peered":"2024-04-12T19:50:24.674820+0000","last_unstale":"2024-04-12T19:50:25.675845+0000","last_undegraded":"2024-04-12T19:50:25.675845+0000","last_fullsized":"2024-04-12T19:50:25.675845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688640+0000","last_change":"2024-04-12T19:50:24.688597+0000","last_active":"2024-04-12T19:50:25.688640+0000","last_peered":"2024-04-12T19:50:25.688640+0000","last_clean":"2024-04-12T19:50:25.688640+0000","last_became_active":"2024-04-12T19:50:24.688308+0000","last_became_peered":"2024-04-12T19:50:24.688308+0000","last_unstale":"2024-04-12T19:50:25.688640+0000","last_undegraded":"2024-04-12T19:50:25.688640+0000","last_fullsized":"2024-04-12T19:50:25.688640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675161+0000","last_change":"2024-04-12T19:50:24.680554+0000","last_active":"2024-04-12T19:50:25.675161+0000","last_peered":"2024-04-12T19:50:25.675161+0000","last_clean":"2024-04-12T19:50:25.675161+0000","last_became_active":"2024-04-12T19:50:24.675178+0000","last_became_peered":"2024-04-12T19:50:24.675178+0000","last_unstale":"2024-04-12T19:50:25.675161+0000","last_undegraded":"2024-04-12T19:50:25.675161+0000","last_fullsized":"2024-04-12T19:50:25.675161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688158+0000","last_change":"2024-04-12T19:50:24.677924+0000","last_active":"2024-04-12T19:50:25.688158+0000","last_peered":"2024-04-12T19:50:25.688158+0000","last_clean":"2024-04-12T19:50:25.688158+0000","last_became_active":"2024-04-12T19:50:24.677414+0000","last_became_peered":"2024-04-12T19:50:24.677414+0000","last_unstale":"2024-04-12T19:50:25.688158+0000","last_undegraded":"2024-04-12T19:50:25.688158+0000","last_fullsized":"2024-04-12T19:50:25.688158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193170+0000","last_change":"2024-04-12T19:50:24.690479+0000","last_active":"2024-04-12T19:50:26.193170+0000","last_peered":"2024-04-12T19:50:26.193170+0000","last_clean":"2024-04-12T19:50:26.193170+0000","last_became_active":"2024-04-12T19:50:24.690010+0000","last_became_peered":"2024-04-12T19:50:24.690010+0000","last_unstale":"2024-04-12T19:50:26.193170+0000","last_undegraded":"2024-04-12T19:50:26.193170+0000","last_fullsized":"2024-04-12T19:50:26.193170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673588+0000","last_change":"2024-04-12T19:50:24.679542+0000","last_active":"2024-04-12T19:50:25.673588+0000","last_peered":"2024-04-12T19:50:25.673588+0000","last_clean":"2024-04-12T19:50:25.673588+0000","last_became_active":"2024-04-12T19:50:24.678975+0000","last_became_peered":"2024-04-12T19:50:24.678975+0000","last_unstale":"2024-04-12T19:50:25.673588+0000","last_undegraded":"2024-04-12T19:50:25.673588+0000","last_fullsized":"2024-04-12T19:50:25.673588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688977+0000","last_change":"2024-04-12T19:50:24.681572+0000","last_active":"2024-04-12T19:50:25.688977+0000","last_peered":"2024-04-12T19:50:25.688977+0000","last_clean":"2024-04-12T19:50:25.688977+0000","last_became_active":"2024-04-12T19:50:24.681309+0000","last_became_peered":"2024-04-12T19:50:24.681309+0000","last_unstale":"2024-04-12T19:50:25.688977+0000","last_undegraded":"2024-04-12T19:50:25.688977+0000","last_fullsized":"2024-04-12T19:50:25.688977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688257+0000","last_change":"2024-04-12T19:50:24.678439+0000","last_active":"2024-04-12T19:50:25.688257+0000","last_peered":"2024-04-12T19:50:25.688257+0000","last_clean":"2024-04-12T19:50:25.688257+0000","last_became_active":"2024-04-12T19:50:24.677934+0000","last_became_peered":"2024-04-12T19:50:24.677934+0000","last_unstale":"2024-04-12T19:50:25.688257+0000","last_undegraded":"2024-04-12T19:50:25.688257+0000","last_fullsized":"2024-04-12T19:50:25.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688579+0000","last_change":"2024-04-12T19:50:24.683054+0000","last_active":"2024-04-12T19:50:25.688579+0000","last_peered":"2024-04-12T19:50:25.688579+0000","last_clean":"2024-04-12T19:50:25.688579+0000","last_became_active":"2024-04-12T19:50:24.682749+0000","last_became_peered":"2024-04-12T19:50:24.682749+0000","last_unstale":"2024-04-12T19:50:25.688579+0000","last_undegraded":"2024-04-12T19:50:25.688579+0000","last_fullsized":"2024-04-12T19:50:25.688579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675969+0000","last_change":"2024-04-12T19:50:24.700812+0000","last_active":"2024-04-12T19:50:25.675969+0000","last_peered":"2024-04-12T19:50:25.675969+0000","last_clean":"2024-04-12T19:50:25.675969+0000","last_became_active":"2024-04-12T19:50:24.700470+0000","last_became_peered":"2024-04-12T19:50:24.700470+0000","last_unstale":"2024-04-12T19:50:25.675969+0000","last_undegraded":"2024-04-12T19:50:25.675969+0000","last_fullsized":"2024-04-12T19:50:25.675969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676868+0000","last_change":"2024-04-12T19:50:24.678150+0000","last_active":"2024-04-12T19:50:25.676868+0000","last_peered":"2024-04-12T19:50:25.676868+0000","last_clean":"2024-04-12T19:50:25.676868+0000","last_became_active":"2024-04-12T19:50:24.677811+0000","last_became_peered":"2024-04-12T19:50:24.677811+0000","last_unstale":"2024-04-12T19:50:25.676868+0000","last_undegraded":"2024-04-12T19:50:25.676868+0000","last_fullsized":"2024-04-12T19:50:25.676868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193225+0000","last_change":"2024-04-12T19:50:24.701334+0000","last_active":"2024-04-12T19:50:26.193225+0000","last_peered":"2024-04-12T19:50:26.193225+0000","last_clean":"2024-04-12T19:50:26.193225+0000","last_became_active":"2024-04-12T19:50:24.701046+0000","last_became_peered":"2024-04-12T19:50:24.701046+0000","last_unstale":"2024-04-12T19:50:26.193225+0000","last_undegraded":"2024-04-12T19:50:26.193225+0000","last_fullsized":"2024-04-12T19:50:26.193225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184164+0000","last_change":"2024-04-12T19:50:24.678787+0000","last_active":"2024-04-12T19:50:26.184164+0000","last_peered":"2024-04-12T19:50:26.184164+0000","last_clean":"2024-04-12T19:50:26.184164+0000","last_became_active":"2024-04-12T19:50:24.678537+0000","last_became_peered":"2024-04-12T19:50:24.678537+0000","last_unstale":"2024-04-12T19:50:26.184164+0000","last_undegraded":"2024-04-12T19:50:26.184164+0000","last_fullsized":"2024-04-12T19:50:26.184164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185746+0000","last_change":"2024-04-12T19:50:24.680249+0000","last_active":"2024-04-12T19:50:26.185746+0000","last_peered":"2024-04-12T19:50:26.185746+0000","last_clean":"2024-04-12T19:50:26.185746+0000","last_became_active":"2024-04-12T19:50:24.679835+0000","last_became_peered":"2024-04-12T19:50:24.679835+0000","last_unstale":"2024-04-12T19:50:26.185746+0000","last_undegraded":"2024-04-12T19:50:26.185746+0000","last_fullsized":"2024-04-12T19:50:26.185746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:17:07.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.750013+0000","last_change":"2024-04-12T19:50:24.697806+0000","last_active":"2024-04-12T19:50:25.750013+0000","last_peered":"2024-04-12T19:50:25.750013+0000","last_clean":"2024-04-12T19:50:25.750013+0000","last_became_active":"2024-04-12T19:50:24.697515+0000","last_became_peered":"2024-04-12T19:50:24.697515+0000","last_unstale":"2024-04-12T19:50:25.750013+0000","last_undegraded":"2024-04-12T19:50:25.750013+0000","last_fullsized":"2024-04-12T19:50:25.750013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:12:47.126078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185806+0000","last_change":"2024-04-12T19:50:24.684475+0000","last_active":"2024-04-12T19:50:26.185806+0000","last_peered":"2024-04-12T19:50:26.185806+0000","last_clean":"2024-04-12T19:50:26.185806+0000","last_became_active":"2024-04-12T19:50:24.684052+0000","last_became_peered":"2024-04-12T19:50:24.684052+0000","last_unstale":"2024-04-12T19:50:26.185806+0000","last_undegraded":"2024-04-12T19:50:26.185806+0000","last_fullsized":"2024-04-12T19:50:26.185806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:19:38.175136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688476+0000","last_change":"2024-04-12T19:50:24.679515+0000","last_active":"2024-04-12T19:50:25.688476+0000","last_peered":"2024-04-12T19:50:25.688476+0000","last_clean":"2024-04-12T19:50:25.688476+0000","last_became_active":"2024-04-12T19:50:24.678757+0000","last_became_peered":"2024-04-12T19:50:24.678757+0000","last_unstale":"2024-04-12T19:50:25.688476+0000","last_undegraded":"2024-04-12T19:50:25.688476+0000","last_fullsized":"2024-04-12T19:50:25.688476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682423+0000","last_change":"2024-04-12T19:50:24.674490+0000","last_active":"2024-04-12T19:50:25.682423+0000","last_peered":"2024-04-12T19:50:25.682423+0000","last_clean":"2024-04-12T19:50:25.682423+0000","last_became_active":"2024-04-12T19:50:24.674131+0000","last_became_peered":"2024-04-12T19:50:24.674131+0000","last_unstale":"2024-04-12T19:50:25.682423+0000","last_undegraded":"2024-04-12T19:50:25.682423+0000","last_fullsized":"2024-04-12T19:50:25.682423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673126+0000","last_change":"2024-04-12T19:50:24.673011+0000","last_active":"2024-04-12T19:50:25.673126+0000","last_peered":"2024-04-12T19:50:25.673126+0000","last_clean":"2024-04-12T19:50:25.673126+0000","last_became_active":"2024-04-12T19:50:24.672490+0000","last_became_peered":"2024-04-12T19:50:24.672490+0000","last_unstale":"2024-04-12T19:50:25.673126+0000","last_undegraded":"2024-04-12T19:50:25.673126+0000","last_fullsized":"2024-04-12T19:50:25.673126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193541+0000","last_change":"2024-04-12T19:50:24.692657+0000","last_active":"2024-04-12T19:50:26.193541+0000","last_peered":"2024-04-12T19:50:26.193541+0000","last_clean":"2024-04-12T19:50:26.193541+0000","last_became_active":"2024-04-12T19:50:24.692335+0000","last_became_peered":"2024-04-12T19:50:24.692335+0000","last_unstale":"2024-04-12T19:50:26.193541+0000","last_undegraded":"2024-04-12T19:50:26.193541+0000","last_fullsized":"2024-04-12T19:50:26.193541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184179+0000","last_change":"2024-04-12T19:50:24.692524+0000","last_active":"2024-04-12T19:50:26.184179+0000","last_peered":"2024-04-12T19:50:26.184179+0000","last_clean":"2024-04-12T19:50:26.184179+0000","last_became_active":"2024-04-12T19:50:24.692239+0000","last_became_peered":"2024-04-12T19:50:24.692239+0000","last_unstale":"2024-04-12T19:50:26.184179+0000","last_undegraded":"2024-04-12T19:50:26.184179+0000","last_fullsized":"2024-04-12T19:50:26.184179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:50:25.705510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193671+0000","last_change":"2024-04-12T19:50:24.686815+0000","last_active":"2024-04-12T19:50:26.193671+0000","last_peered":"2024-04-12T19:50:26.193671+0000","last_clean":"2024-04-12T19:50:26.193671+0000","last_became_active":"2024-04-12T19:50:24.686192+0000","last_became_peered":"2024-04-12T19:50:24.686192+0000","last_unstale":"2024-04-12T19:50:26.193671+0000","last_undegraded":"2024-04-12T19:50:26.193671+0000","last_fullsized":"2024-04-12T19:50:26.193671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676094+0000","last_change":"2024-04-12T19:50:24.686489+0000","last_active":"2024-04-12T19:50:25.676094+0000","last_peered":"2024-04-12T19:50:25.676094+0000","last_clean":"2024-04-12T19:50:25.676094+0000","last_became_active":"2024-04-12T19:50:24.685753+0000","last_became_peered":"2024-04-12T19:50:24.685753+0000","last_unstale":"2024-04-12T19:50:25.676094+0000","last_undegraded":"2024-04-12T19:50:25.676094+0000","last_fullsized":"2024-04-12T19:50:25.676094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676600+0000","last_change":"2024-04-12T19:50:24.675093+0000","last_active":"2024-04-12T19:50:25.676600+0000","last_peered":"2024-04-12T19:50:25.676600+0000","last_clean":"2024-04-12T19:50:25.676600+0000","last_became_active":"2024-04-12T19:50:24.674798+0000","last_became_peered":"2024-04-12T19:50:24.674798+0000","last_unstale":"2024-04-12T19:50:25.676600+0000","last_undegraded":"2024-04-12T19:50:25.676600+0000","last_fullsized":"2024-04-12T19:50:25.676600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:07.561061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676605+0000","last_change":"2024-04-12T19:50:24.681961+0000","last_active":"2024-04-12T19:50:25.676605+0000","last_peered":"2024-04-12T19:50:25.676605+0000","last_clean":"2024-04-12T19:50:25.676605+0000","last_became_active":"2024-04-12T19:50:24.681596+0000","last_became_peered":"2024-04-12T19:50:24.681596+0000","last_unstale":"2024-04-12T19:50:25.676605+0000","last_undegraded":"2024-04-12T19:50:25.676605+0000","last_fullsized":"2024-04-12T19:50:25.676605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194168+0000","last_change":"2024-04-12T19:50:24.688114+0000","last_active":"2024-04-12T19:50:26.194168+0000","last_peered":"2024-04-12T19:50:26.194168+0000","last_clean":"2024-04-12T19:50:26.194168+0000","last_became_active":"2024-04-12T19:50:24.687826+0000","last_became_peered":"2024-04-12T19:50:24.687826+0000","last_unstale":"2024-04-12T19:50:26.194168+0000","last_undegraded":"2024-04-12T19:50:26.194168+0000","last_fullsized":"2024-04-12T19:50:26.194168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:36:03.537218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184379+0000","last_change":"2024-04-12T19:50:24.697023+0000","last_active":"2024-04-12T19:50:26.184379+0000","last_peered":"2024-04-12T19:50:26.184379+0000","last_clean":"2024-04-12T19:50:26.184379+0000","last_became_active":"2024-04-12T19:50:24.696739+0000","last_became_peered":"2024-04-12T19:50:24.696739+0000","last_unstale":"2024-04-12T19:50:26.184379+0000","last_undegraded":"2024-04-12T19:50:26.184379+0000","last_fullsized":"2024-04-12T19:50:26.184379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:46:45.869946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749594+0000","last_change":"2024-04-12T19:50:24.675081+0000","last_active":"2024-04-12T19:50:25.749594+0000","last_peered":"2024-04-12T19:50:25.749594+0000","last_clean":"2024-04-12T19:50:25.749594+0000","last_became_active":"2024-04-12T19:50:24.674812+0000","last_became_peered":"2024-04-12T19:50:24.674812+0000","last_unstale":"2024-04-12T19:50:25.749594+0000","last_undegraded":"2024-04-12T19:50:25.749594+0000","last_fullsized":"2024-04-12T19:50:25.749594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:51:04.796842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.674913+0000","last_change":"2024-04-12T19:50:24.687854+0000","last_active":"2024-04-12T19:50:25.674913+0000","last_peered":"2024-04-12T19:50:25.674913+0000","last_clean":"2024-04-12T19:50:25.674913+0000","last_became_active":"2024-04-12T19:50:24.685605+0000","last_became_peered":"2024-04-12T19:50:24.685605+0000","last_unstale":"2024-04-12T19:50:25.674913+0000","last_undegraded":"2024-04-12T19:50:25.674913+0000","last_fullsized":"2024-04-12T19:50:25.674913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:11.962955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673270+0000","last_change":"2024-04-12T19:50:24.694100+0000","last_active":"2024-04-12T19:50:25.673270+0000","last_peered":"2024-04-12T19:50:25.673270+0000","last_clean":"2024-04-12T19:50:25.673270+0000","last_became_active":"2024-04-12T19:50:24.693828+0000","last_became_peered":"2024-04-12T19:50:24.693828+0000","last_unstale":"2024-04-12T19:50:25.673270+0000","last_undegraded":"2024-04-12T19:50:25.673270+0000","last_fullsized":"2024-04-12T19:50:25.673270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:56.341338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194122+0000","last_change":"2024-04-12T19:50:23.685737+0000","last_active":"2024-04-12T19:50:26.194122+0000","last_peered":"2024-04-12T19:50:26.194122+0000","last_clean":"2024-04-12T19:50:26.194122+0000","last_became_active":"2024-04-12T19:50:23.685371+0000","last_became_peered":"2024-04-12T19:50:23.685371+0000","last_unstale":"2024-04-12T19:50:26.194122+0000","last_undegraded":"2024-04-12T19:50:26.194122+0000","last_fullsized":"2024-04-12T19:50:26.194122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682544+0000","last_change":"2024-04-12T19:50:24.681008+0000","last_active":"2024-04-12T19:50:25.682544+0000","last_peered":"2024-04-12T19:50:25.682544+0000","last_clean":"2024-04-12T19:50:25.682544+0000","last_became_active":"2024-04-12T19:50:24.680689+0000","last_became_peered":"2024-04-12T19:50:24.680689+0000","last_unstale":"2024-04-12T19:50:25.682544+0000","last_undegraded":"2024-04-12T19:50:25.682544+0000","last_fullsized":"2024-04-12T19:50:25.682544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:16:49.798144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184687+0000","last_change":"2024-04-12T19:50:23.691933+0000","last_active":"2024-04-12T19:50:26.184687+0000","last_peered":"2024-04-12T19:50:26.184687+0000","last_clean":"2024-04-12T19:50:26.184687+0000","last_became_active":"2024-04-12T19:50:23.691622+0000","last_became_peered":"2024-04-12T19:50:23.691622+0000","last_unstale":"2024-04-12T19:50:26.184687+0000","last_undegraded":"2024-04-12T19:50:26.184687+0000","last_fullsized":"2024-04-12T19:50:26.184687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675732+0000","last_change":"2024-04-12T19:50:23.691962+0000","last_active":"2024-04-12T19:50:25.675732+0000","last_peered":"2024-04-12T19:50:25.675732+0000","last_clean":"2024-04-12T19:50:25.675732+0000","last_became_active":"2024-04-12T19:50:23.691571+0000","last_became_peered":"2024-04-12T19:50:23.691571+0000","last_unstale":"2024-04-12T19:50:25.675732+0000","last_undegraded":"2024-04-12T19:50:25.675732+0000","last_fullsized":"2024-04-12T19:50:25.675732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:10:39.564249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193629+0000","last_change":"2024-04-12T19:50:24.690993+0000","last_active":"2024-04-12T19:50:26.193629+0000","last_peered":"2024-04-12T19:50:26.193629+0000","last_clean":"2024-04-12T19:50:26.193629+0000","last_became_active":"2024-04-12T19:50:24.690721+0000","last_became_peered":"2024-04-12T19:50:24.690721+0000","last_unstale":"2024-04-12T19:50:26.193629+0000","last_undegraded":"2024-04-12T19:50:26.193629+0000","last_fullsized":"2024-04-12T19:50:26.193629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:33:38.745278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184883+0000","last_change":"2024-04-12T19:50:23.689397+0000","last_active":"2024-04-12T19:50:26.184883+0000","last_peered":"2024-04-12T19:50:26.184883+0000","last_clean":"2024-04-12T19:50:26.184883+0000","last_became_active":"2024-04-12T19:50:23.688493+0000","last_became_peered":"2024-04-12T19:50:23.688493+0000","last_unstale":"2024-04-12T19:50:26.184883+0000","last_undegraded":"2024-04-12T19:50:26.184883+0000","last_fullsized":"2024-04-12T19:50:26.184883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682299+0000","last_change":"2024-04-12T19:50:24.667019+0000","last_active":"2024-04-12T19:50:25.682299+0000","last_peered":"2024-04-12T19:50:25.682299+0000","last_clean":"2024-04-12T19:50:25.682299+0000","last_became_active":"2024-04-12T19:50:24.666763+0000","last_became_peered":"2024-04-12T19:50:24.666763+0000","last_unstale":"2024-04-12T19:50:25.682299+0000","last_undegraded":"2024-04-12T19:50:25.682299+0000","last_fullsized":"2024-04-12T19:50:25.682299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:52:02.072564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673739+0000","last_change":"2024-04-12T19:50:23.686496+0000","last_active":"2024-04-12T19:50:25.673739+0000","last_peered":"2024-04-12T19:50:25.673739+0000","last_clean":"2024-04-12T19:50:25.673739+0000","last_became_active":"2024-04-12T19:50:23.684913+0000","last_became_peered":"2024-04-12T19:50:23.684913+0000","last_unstale":"2024-04-12T19:50:25.673739+0000","last_undegraded":"2024-04-12T19:50:25.673739+0000","last_fullsized":"2024-04-12T19:50:25.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676067+0000","last_change":"2024-04-12T19:50:24.693409+0000","last_active":"2024-04-12T19:50:25.676067+0000","last_peered":"2024-04-12T19:50:25.676067+0000","last_clean":"2024-04-12T19:50:25.676067+0000","last_became_active":"2024-04-12T19:50:24.686066+0000","last_became_peered":"2024-04-12T19:50:24.686066+0000","last_unstale":"2024-04-12T19:50:25.676067+0000","last_undegraded":"2024-04-12T19:50:25.676067+0000","last_fullsized":"2024-04-12T19:50:25.676067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689282+0000","last_change":"2024-04-12T19:50:23.685660+0000","last_active":"2024-04-12T19:50:25.689282+0000","last_peered":"2024-04-12T19:50:25.689282+0000","last_clean":"2024-04-12T19:50:25.689282+0000","last_became_active":"2024-04-12T19:50:23.685112+0000","last_became_peered":"2024-04-12T19:50:23.685112+0000","last_unstale":"2024-04-12T19:50:25.689282+0000","last_undegraded":"2024-04-12T19:50:25.689282+0000","last_fullsized":"2024-04-12T19:50:25.689282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194160+0000","last_change":"2024-04-12T19:50:24.687686+0000","last_active":"2024-04-12T19:50:26.194160+0000","last_peered":"2024-04-12T19:50:26.194160+0000","last_clean":"2024-04-12T19:50:26.194160+0000","last_became_active":"2024-04-12T19:50:24.686557+0000","last_became_peered":"2024-04-12T19:50:24.686557+0000","last_unstale":"2024-04-12T19:50:26.194160+0000","last_undegraded":"2024-04-12T19:50:26.194160+0000","last_fullsized":"2024-04-12T19:50:26.194160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:51:33.330599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749850+0000","last_change":"2024-04-12T19:50:23.688709+0000","last_active":"2024-04-12T19:50:25.749850+0000","last_peered":"2024-04-12T19:50:25.749850+0000","last_clean":"2024-04-12T19:50:25.749850+0000","last_became_active":"2024-04-12T19:50:23.687448+0000","last_became_peered":"2024-04-12T19:50:23.687448+0000","last_unstale":"2024-04-12T19:50:25.749850+0000","last_undegraded":"2024-04-12T19:50:25.749850+0000","last_fullsized":"2024-04-12T19:50:25.749850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:41.154672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184609+0000","last_change":"2024-04-12T19:50:24.692484+0000","last_active":"2024-04-12T19:50:26.184609+0000","last_peered":"2024-04-12T19:50:26.184609+0000","last_clean":"2024-04-12T19:50:26.184609+0000","last_became_active":"2024-04-12T19:50:24.692157+0000","last_became_peered":"2024-04-12T19:50:24.692157+0000","last_unstale":"2024-04-12T19:50:26.184609+0000","last_undegraded":"2024-04-12T19:50:26.184609+0000","last_fullsized":"2024-04-12T19:50:26.184609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:51:48.663863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675846+0000","last_change":"2024-04-12T19:50:23.686058+0000","last_active":"2024-04-12T19:50:25.675846+0000","last_peered":"2024-04-12T19:50:25.675846+0000","last_clean":"2024-04-12T19:50:25.675846+0000","last_became_active":"2024-04-12T19:50:23.685765+0000","last_became_peered":"2024-04-12T19:50:23.685765+0000","last_unstale":"2024-04-12T19:50:25.675846+0000","last_undegraded":"2024-04-12T19:50:25.675846+0000","last_fullsized":"2024-04-12T19:50:25.675846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:04:17.211024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688457+0000","last_change":"2024-04-12T19:50:24.682664+0000","last_active":"2024-04-12T19:50:25.688457+0000","last_peered":"2024-04-12T19:50:25.688457+0000","last_clean":"2024-04-12T19:50:25.688457+0000","last_became_active":"2024-04-12T19:50:24.682385+0000","last_became_peered":"2024-04-12T19:50:24.682385+0000","last_unstale":"2024-04-12T19:50:25.688457+0000","last_undegraded":"2024-04-12T19:50:25.688457+0000","last_fullsized":"2024-04-12T19:50:25.688457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:30:39.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749812+0000","last_change":"2024-04-12T19:50:23.689886+0000","last_active":"2024-04-12T19:50:25.749812+0000","last_peered":"2024-04-12T19:50:25.749812+0000","last_clean":"2024-04-12T19:50:25.749812+0000","last_became_active":"2024-04-12T19:50:23.689536+0000","last_became_peered":"2024-04-12T19:50:23.689536+0000","last_unstale":"2024-04-12T19:50:25.749812+0000","last_undegraded":"2024-04-12T19:50:25.749812+0000","last_fullsized":"2024-04-12T19:50:25.749812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:30.521133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675832+0000","last_change":"2024-04-12T19:50:24.689487+0000","last_active":"2024-04-12T19:50:25.675832+0000","last_peered":"2024-04-12T19:50:25.675832+0000","last_clean":"2024-04-12T19:50:25.675832+0000","last_became_active":"2024-04-12T19:50:24.687365+0000","last_became_peered":"2024-04-12T19:50:24.687365+0000","last_unstale":"2024-04-12T19:50:25.675832+0000","last_undegraded":"2024-04-12T19:50:25.675832+0000","last_fullsized":"2024-04-12T19:50:25.675832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:30.190677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184932+0000","last_change":"2024-04-12T19:50:23.682121+0000","last_active":"2024-04-12T19:50:26.184932+0000","last_peered":"2024-04-12T19:50:26.184932+0000","last_clean":"2024-04-12T19:50:26.184932+0000","last_became_active":"2024-04-12T19:50:23.681803+0000","last_became_peered":"2024-04-12T19:50:23.681803+0000","last_unstale":"2024-04-12T19:50:26.184932+0000","last_undegraded":"2024-04-12T19:50:26.184932+0000","last_fullsized":"2024-04-12T19:50:26.184932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:55.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184717+0000","last_change":"2024-04-12T19:50:24.685212+0000","last_active":"2024-04-12T19:50:26.184717+0000","last_peered":"2024-04-12T19:50:26.184717+0000","last_clean":"2024-04-12T19:50:26.184717+0000","last_became_active":"2024-04-12T19:50:24.684863+0000","last_became_peered":"2024-04-12T19:50:24.684863+0000","last_unstale":"2024-04-12T19:50:26.184717+0000","last_undegraded":"2024-04-12T19:50:26.184717+0000","last_fullsized":"2024-04-12T19:50:26.184717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:36:27.562288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687944+0000","last_change":"2024-04-12T19:50:23.673596+0000","last_active":"2024-04-12T19:50:25.687944+0000","last_peered":"2024-04-12T19:50:25.687944+0000","last_clean":"2024-04-12T19:50:25.687944+0000","last_became_active":"2024-04-12T19:50:23.672686+0000","last_became_peered":"2024-04-12T19:50:23.672686+0000","last_unstale":"2024-04-12T19:50:25.687944+0000","last_undegraded":"2024-04-12T19:50:25.687944+0000","last_fullsized":"2024-04-12T19:50:25.687944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672719+0000","last_change":"2024-04-12T19:50:24.693699+0000","last_active":"2024-04-12T19:50:25.672719+0000","last_peered":"2024-04-12T19:50:25.672719+0000","last_clean":"2024-04-12T19:50:25.672719+0000","last_became_active":"2024-04-12T19:50:24.693376+0000","last_became_peered":"2024-04-12T19:50:24.693376+0000","last_unstale":"2024-04-12T19:50:25.672719+0000","last_undegraded":"2024-04-12T19:50:25.672719+0000","last_fullsized":"2024-04-12T19:50:25.672719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:03.379464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682161+0000","last_change":"2024-04-12T19:50:23.682371+0000","last_active":"2024-04-12T19:50:25.682161+0000","last_peered":"2024-04-12T19:50:25.682161+0000","last_clean":"2024-04-12T19:50:25.682161+0000","last_became_active":"2024-04-12T19:50:23.681683+0000","last_became_peered":"2024-04-12T19:50:23.681683+0000","last_unstale":"2024-04-12T19:50:25.682161+0000","last_undegraded":"2024-04-12T19:50:25.682161+0000","last_fullsized":"2024-04-12T19:50:25.682161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184368+0000","last_change":"2024-04-12T19:50:24.673851+0000","last_active":"2024-04-12T19:50:26.184368+0000","last_peered":"2024-04-12T19:50:26.184368+0000","last_clean":"2024-04-12T19:50:26.184368+0000","last_became_active":"2024-04-12T19:50:24.673551+0000","last_became_peered":"2024-04-12T19:50:24.673551+0000","last_unstale":"2024-04-12T19:50:26.184368+0000","last_undegraded":"2024-04-12T19:50:26.184368+0000","last_fullsized":"2024-04-12T19:50:26.184368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:29:34.148711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689481+0000","last_change":"2024-04-12T19:50:23.691929+0000","last_active":"2024-04-12T19:50:25.689481+0000","last_peered":"2024-04-12T19:50:25.689481+0000","last_clean":"2024-04-12T19:50:25.689481+0000","last_became_active":"2024-04-12T19:50:23.684811+0000","last_became_peered":"2024-04-12T19:50:23.684811+0000","last_unstale":"2024-04-12T19:50:25.689481+0000","last_undegraded":"2024-04-12T19:50:25.689481+0000","last_fullsized":"2024-04-12T19:50:25.689481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676370+0000","last_change":"2024-04-12T19:50:24.691614+0000","last_active":"2024-04-12T19:50:25.676370+0000","last_peered":"2024-04-12T19:50:25.676370+0000","last_clean":"2024-04-12T19:50:25.676370+0000","last_became_active":"2024-04-12T19:50:24.686194+0000","last_became_peered":"2024-04-12T19:50:24.686194+0000","last_unstale":"2024-04-12T19:50:25.676370+0000","last_undegraded":"2024-04-12T19:50:25.676370+0000","last_fullsized":"2024-04-12T19:50:25.676370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:45.278393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683033+0000","last_change":"2024-04-12T19:50:23.685902+0000","last_active":"2024-04-12T19:50:25.683033+0000","last_peered":"2024-04-12T19:50:25.683033+0000","last_clean":"2024-04-12T19:50:25.683033+0000","last_became_active":"2024-04-12T19:50:23.685614+0000","last_became_peered":"2024-04-12T19:50:23.685614+0000","last_unstale":"2024-04-12T19:50:25.683033+0000","last_undegraded":"2024-04-12T19:50:25.683033+0000","last_fullsized":"2024-04-12T19:50:25.683033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.689356+0000","last_change":"2024-04-12T19:50:24.674299+0000","last_active":"2024-04-12T19:50:25.689356+0000","last_peered":"2024-04-12T19:50:25.689356+0000","last_clean":"2024-04-12T19:50:25.689356+0000","last_became_active":"2024-04-12T19:50:24.672616+0000","last_became_peered":"2024-04-12T19:50:24.672616+0000","last_unstale":"2024-04-12T19:50:25.689356+0000","last_undegraded":"2024-04-12T19:50:25.689356+0000","last_fullsized":"2024-04-12T19:50:25.689356+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194553+0000","last_change":"2024-04-12T19:50:23.685829+0000","last_active":"2024-04-12T19:50:26.194553+0000","last_peered":"2024-04-12T19:50:26.194553+0000","last_clean":"2024-04-12T19:50:26.194553+0000","last_became_active":"2024-04-12T19:50:23.684944+0000","last_became_peered":"2024-04-12T19:50:23.684944+0000","last_unstale":"2024-04-12T19:50:26.194553+0000","last_undegraded":"2024-04-12T19:50:26.194553+0000","last_fullsized":"2024-04-12T19:50:26.194553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688739+0000","last_change":"2024-04-12T19:50:24.680192+0000","last_active":"2024-04-12T19:50:25.688739+0000","last_peered":"2024-04-12T19:50:25.688739+0000","last_clean":"2024-04-12T19:50:25.688739+0000","last_became_active":"2024-04-12T19:50:24.679916+0000","last_became_peered":"2024-04-12T19:50:24.679916+0000","last_unstale":"2024-04-12T19:50:25.688739+0000","last_undegraded":"2024-04-12T19:50:25.688739+0000","last_fullsized":"2024-04-12T19:50:25.688739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:36:31.112775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676558+0000","last_change":"2024-04-12T19:50:23.693367+0000","last_active":"2024-04-12T19:50:25.676558+0000","last_peered":"2024-04-12T19:50:25.676558+0000","last_clean":"2024-04-12T19:50:25.676558+0000","last_became_active":"2024-04-12T19:50:23.693036+0000","last_became_peered":"2024-04-12T19:50:23.693036+0000","last_unstale":"2024-04-12T19:50:25.676558+0000","last_undegraded":"2024-04-12T19:50:25.676558+0000","last_fullsized":"2024-04-12T19:50:25.676558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185470+0000","last_change":"2024-04-12T19:50:24.674923+0000","last_active":"2024-04-12T19:50:26.185470+0000","last_peered":"2024-04-12T19:50:26.185470+0000","last_clean":"2024-04-12T19:50:26.185470+0000","last_became_active":"2024-04-12T19:50:24.674524+0000","last_became_peered":"2024-04-12T19:50:24.674524+0000","last_unstale":"2024-04-12T19:50:26.185470+0000","last_undegraded":"2024-04-12T19:50:26.185470+0000","last_fullsized":"2024-04-12T19:50:26.185470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:44:56.439740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.672856+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:50:25.672856+0000","last_peered":"2024-04-12T19:50:25.672856+0000","last_clean":"2024-04-12T19:50:25.672856+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:50:25.672856+0000","last_undegraded":"2024-04-12T19:50:25.672856+0000","last_fullsized":"2024-04-12T19:50:25.672856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184860+0000","last_change":"2024-04-12T19:50:23.676923+0000","last_active":"2024-04-12T19:50:26.184860+0000","last_peered":"2024-04-12T19:50:26.184860+0000","last_clean":"2024-04-12T19:50:26.184860+0000","last_became_active":"2024-04-12T19:50:23.676667+0000","last_became_peered":"2024-04-12T19:50:23.676667+0000","last_unstale":"2024-04-12T19:50:26.184860+0000","last_undegraded":"2024-04-12T19:50:26.184860+0000","last_fullsized":"2024-04-12T19:50:26.184860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:13:29.176211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682938+0000","last_change":"2024-04-12T19:50:24.673694+0000","last_active":"2024-04-12T19:50:25.682938+0000","last_peered":"2024-04-12T19:50:25.682938+0000","last_clean":"2024-04-12T19:50:25.682938+0000","last_became_active":"2024-04-12T19:50:24.673340+0000","last_became_peered":"2024-04-12T19:50:24.673340+0000","last_unstale":"2024-04-12T19:50:25.682938+0000","last_undegraded":"2024-04-12T19:50:25.682938+0000","last_fullsized":"2024-04-12T19:50:25.682938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:16:23.808568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184624+0000","last_change":"2024-04-12T19:50:23.690110+0000","last_active":"2024-04-12T19:50:26.184624+0000","last_peered":"2024-04-12T19:50:26.184624+0000","last_clean":"2024-04-12T19:50:26.184624+0000","last_became_active":"2024-04-12T19:50:23.689677+0000","last_became_peered":"2024-04-12T19:50:23.689677+0000","last_unstale":"2024-04-12T19:50:26.184624+0000","last_undegraded":"2024-04-12T19:50:26.184624+0000","last_fullsized":"2024-04-12T19:50:26.184624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:02.364998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676043+0000","last_change":"2024-04-12T19:50:24.675131+0000","last_active":"2024-04-12T19:50:25.676043+0000","last_peered":"2024-04-12T19:50:25.676043+0000","last_clean":"2024-04-12T19:50:25.676043+0000","last_became_active":"2024-04-12T19:50:24.674486+0000","last_became_peered":"2024-04-12T19:50:24.674486+0000","last_unstale":"2024-04-12T19:50:25.676043+0000","last_undegraded":"2024-04-12T19:50:25.676043+0000","last_fullsized":"2024-04-12T19:50:25.676043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:40.879093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749308+0000","last_change":"2024-04-12T19:50:23.690325+0000","last_active":"2024-04-12T19:50:25.749308+0000","last_peered":"2024-04-12T19:50:25.749308+0000","last_clean":"2024-04-12T19:50:25.749308+0000","last_became_active":"2024-04-12T19:50:23.688951+0000","last_became_peered":"2024-04-12T19:50:23.688951+0000","last_unstale":"2024-04-12T19:50:25.749308+0000","last_undegraded":"2024-04-12T19:50:25.749308+0000","last_fullsized":"2024-04-12T19:50:25.749308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:49:41.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193423+0000","last_change":"2024-04-12T19:50:24.698245+0000","last_active":"2024-04-12T19:50:26.193423+0000","last_peered":"2024-04-12T19:50:26.193423+0000","last_clean":"2024-04-12T19:50:26.193423+0000","last_became_active":"2024-04-12T19:50:24.697937+0000","last_became_peered":"2024-04-12T19:50:24.697937+0000","last_unstale":"2024-04-12T19:50:26.193423+0000","last_undegraded":"2024-04-12T19:50:26.193423+0000","last_fullsized":"2024-04-12T19:50:26.193423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185039+0000","last_change":"2024-04-12T19:50:23.689007+0000","last_active":"2024-04-12T19:50:26.185039+0000","last_peered":"2024-04-12T19:50:26.185039+0000","last_clean":"2024-04-12T19:50:26.185039+0000","last_became_active":"2024-04-12T19:50:23.688654+0000","last_became_peered":"2024-04-12T19:50:23.688654+0000","last_unstale":"2024-04-12T19:50:26.185039+0000","last_undegraded":"2024-04-12T19:50:26.185039+0000","last_fullsized":"2024-04-12T19:50:26.185039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:24.585940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688821+0000","last_change":"2024-04-12T19:50:24.680562+0000","last_active":"2024-04-12T19:50:25.688821+0000","last_peered":"2024-04-12T19:50:25.688821+0000","last_clean":"2024-04-12T19:50:25.688821+0000","last_became_active":"2024-04-12T19:50:24.680276+0000","last_became_peered":"2024-04-12T19:50:24.680276+0000","last_unstale":"2024-04-12T19:50:25.688821+0000","last_undegraded":"2024-04-12T19:50:25.688821+0000","last_fullsized":"2024-04-12T19:50:25.688821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:19:50.633606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.193774+0000","last_change":"2024-04-12T19:50:23.685407+0000","last_active":"2024-04-12T19:50:26.193774+0000","last_peered":"2024-04-12T19:50:26.193774+0000","last_clean":"2024-04-12T19:50:26.193774+0000","last_became_active":"2024-04-12T19:50:23.684661+0000","last_became_peered":"2024-04-12T19:50:23.684661+0000","last_unstale":"2024-04-12T19:50:26.193774+0000","last_undegraded":"2024-04-12T19:50:26.193774+0000","last_fullsized":"2024-04-12T19:50:26.193774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:34:37.170442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.675625+0000","last_change":"2024-04-12T19:50:24.686111+0000","last_active":"2024-04-12T19:50:25.675625+0000","last_peered":"2024-04-12T19:50:25.675625+0000","last_clean":"2024-04-12T19:50:25.675625+0000","last_became_active":"2024-04-12T19:50:24.685534+0000","last_became_peered":"2024-04-12T19:50:24.685534+0000","last_unstale":"2024-04-12T19:50:25.675625+0000","last_undegraded":"2024-04-12T19:50:25.675625+0000","last_fullsized":"2024-04-12T19:50:25.675625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:07.387428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185739+0000","last_change":"2024-04-12T19:50:23.676900+0000","last_active":"2024-04-12T19:50:26.185739+0000","last_peered":"2024-04-12T19:50:26.185739+0000","last_clean":"2024-04-12T19:50:26.185739+0000","last_became_active":"2024-04-12T19:50:23.676612+0000","last_became_peered":"2024-04-12T19:50:23.676612+0000","last_unstale":"2024-04-12T19:50:26.185739+0000","last_undegraded":"2024-04-12T19:50:26.185739+0000","last_fullsized":"2024-04-12T19:50:26.185739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:43:39.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676360+0000","last_change":"2024-04-12T19:50:24.695675+0000","last_active":"2024-04-12T19:50:25.676360+0000","last_peered":"2024-04-12T19:50:25.676360+0000","last_clean":"2024-04-12T19:50:25.676360+0000","last_became_active":"2024-04-12T19:50:24.695245+0000","last_became_peered":"2024-04-12T19:50:24.695245+0000","last_unstale":"2024-04-12T19:50:25.676360+0000","last_undegraded":"2024-04-12T19:50:25.676360+0000","last_fullsized":"2024-04-12T19:50:25.676360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.686136+0000","last_change":"2024-04-12T19:50:23.681572+0000","last_active":"2024-04-12T19:50:25.686136+0000","last_peered":"2024-04-12T19:50:25.686136+0000","last_clean":"2024-04-12T19:50:25.686136+0000","last_became_active":"2024-04-12T19:50:23.679612+0000","last_became_peered":"2024-04-12T19:50:23.679612+0000","last_unstale":"2024-04-12T19:50:25.686136+0000","last_undegraded":"2024-04-12T19:50:25.686136+0000","last_fullsized":"2024-04-12T19:50:25.686136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:25:28.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682950+0000","last_change":"2024-04-12T19:50:24.686596+0000","last_active":"2024-04-12T19:50:25.682950+0000","last_peered":"2024-04-12T19:50:25.682950+0000","last_clean":"2024-04-12T19:50:25.682950+0000","last_became_active":"2024-04-12T19:50:24.685023+0000","last_became_peered":"2024-04-12T19:50:24.685023+0000","last_unstale":"2024-04-12T19:50:25.682950+0000","last_undegraded":"2024-04-12T19:50:25.682950+0000","last_fullsized":"2024-04-12T19:50:25.682950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:14:38.661839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185887+0000","last_change":"2024-04-12T19:50:23.684680+0000","last_active":"2024-04-12T19:50:26.185887+0000","last_peered":"2024-04-12T19:50:26.185887+0000","last_clean":"2024-04-12T19:50:26.185887+0000","last_became_active":"2024-04-12T19:50:23.684298+0000","last_became_peered":"2024-04-12T19:50:23.684298+0000","last_unstale":"2024-04-12T19:50:26.185887+0000","last_undegraded":"2024-04-12T19:50:26.185887+0000","last_fullsized":"2024-04-12T19:50:26.185887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:18:06.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683078+0000","last_change":"2024-04-12T19:50:24.676462+0000","last_active":"2024-04-12T19:50:25.683078+0000","last_peered":"2024-04-12T19:50:25.683078+0000","last_clean":"2024-04-12T19:50:25.683078+0000","last_became_active":"2024-04-12T19:50:24.676182+0000","last_became_peered":"2024-04-12T19:50:24.676182+0000","last_unstale":"2024-04-12T19:50:25.683078+0000","last_undegraded":"2024-04-12T19:50:25.683078+0000","last_fullsized":"2024-04-12T19:50:25.683078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:42:20.689869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.687692+0000","last_change":"2024-04-12T19:50:23.684797+0000","last_active":"2024-04-12T19:50:25.687692+0000","last_peered":"2024-04-12T19:50:25.687692+0000","last_clean":"2024-04-12T19:50:25.687692+0000","last_became_active":"2024-04-12T19:50:23.678108+0000","last_became_peered":"2024-04-12T19:50:23.678108+0000","last_unstale":"2024-04-12T19:50:25.687692+0000","last_undegraded":"2024-04-12T19:50:25.687692+0000","last_fullsized":"2024-04-12T19:50:25.687692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184277+0000","last_change":"2024-04-12T19:50:24.681150+0000","last_active":"2024-04-12T19:50:26.184277+0000","last_peered":"2024-04-12T19:50:26.184277+0000","last_clean":"2024-04-12T19:50:26.184277+0000","last_became_active":"2024-04-12T19:50:24.680857+0000","last_became_peered":"2024-04-12T19:50:24.680857+0000","last_unstale":"2024-04-12T19:50:26.184277+0000","last_undegraded":"2024-04-12T19:50:26.184277+0000","last_fullsized":"2024-04-12T19:50:26.184277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:07:34.111751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682575+0000","last_change":"2024-04-12T19:50:23.691210+0000","last_active":"2024-04-12T19:50:25.682575+0000","last_peered":"2024-04-12T19:50:25.682575+0000","last_clean":"2024-04-12T19:50:25.682575+0000","last_became_active":"2024-04-12T19:50:23.690943+0000","last_became_peered":"2024-04-12T19:50:23.690943+0000","last_unstale":"2024-04-12T19:50:25.682575+0000","last_undegraded":"2024-04-12T19:50:25.682575+0000","last_fullsized":"2024-04-12T19:50:25.682575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:47:56.579140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.682186+0000","last_change":"2024-04-12T19:50:24.667678+0000","last_active":"2024-04-12T19:50:25.682186+0000","last_peered":"2024-04-12T19:50:25.682186+0000","last_clean":"2024-04-12T19:50:25.682186+0000","last_became_active":"2024-04-12T19:50:24.667407+0000","last_became_peered":"2024-04-12T19:50:24.667407+0000","last_unstale":"2024-04-12T19:50:25.682186+0000","last_undegraded":"2024-04-12T19:50:25.682186+0000","last_fullsized":"2024-04-12T19:50:25.682186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:46:48.592456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.676272+0000","last_change":"2024-04-12T19:50:23.690317+0000","last_active":"2024-04-12T19:50:25.676272+0000","last_peered":"2024-04-12T19:50:25.676272+0000","last_clean":"2024-04-12T19:50:25.676272+0000","last_became_active":"2024-04-12T19:50:23.689037+0000","last_became_peered":"2024-04-12T19:50:23.689037+0000","last_unstale":"2024-04-12T19:50:25.676272+0000","last_undegraded":"2024-04-12T19:50:25.676272+0000","last_fullsized":"2024-04-12T19:50:25.676272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:46:42.782745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.749840+0000","last_change":"2024-04-12T19:50:24.681363+0000","last_active":"2024-04-12T19:50:25.749840+0000","last_peered":"2024-04-12T19:50:25.749840+0000","last_clean":"2024-04-12T19:50:25.749840+0000","last_became_active":"2024-04-12T19:50:24.681100+0000","last_became_peered":"2024-04-12T19:50:24.681100+0000","last_unstale":"2024-04-12T19:50:25.749840+0000","last_undegraded":"2024-04-12T19:50:25.749840+0000","last_fullsized":"2024-04-12T19:50:25.749840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:04:43.842205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184678+0000","last_change":"2024-04-12T19:50:23.679805+0000","last_active":"2024-04-12T19:50:26.184678+0000","last_peered":"2024-04-12T19:50:26.184678+0000","last_clean":"2024-04-12T19:50:26.184678+0000","last_became_active":"2024-04-12T19:50:23.679501+0000","last_became_peered":"2024-04-12T19:50:23.679501+0000","last_unstale":"2024-04-12T19:50:26.184678+0000","last_undegraded":"2024-04-12T19:50:26.184678+0000","last_fullsized":"2024-04-12T19:50:26.184678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:55:44.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.184641+0000","last_change":"2024-04-12T19:50:24.678712+0000","last_active":"2024-04-12T19:50:26.184641+0000","last_peered":"2024-04-12T19:50:26.184641+0000","last_clean":"2024-04-12T19:50:26.184641+0000","last_became_active":"2024-04-12T19:50:24.677584+0000","last_became_peered":"2024-04-12T19:50:24.677584+0000","last_unstale":"2024-04-12T19:50:26.184641+0000","last_undegraded":"2024-04-12T19:50:26.184641+0000","last_fullsized":"2024-04-12T19:50:26.184641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:27:08.109541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.185338+0000","last_change":"2024-04-12T19:50:24.698869+0000","last_active":"2024-04-12T19:50:26.185338+0000","last_peered":"2024-04-12T19:50:26.185338+0000","last_clean":"2024-04-12T19:50:26.185338+0000","last_became_active":"2024-04-12T19:50:24.698605+0000","last_became_peered":"2024-04-12T19:50:24.698605+0000","last_unstale":"2024-04-12T19:50:26.185338+0000","last_undegraded":"2024-04-12T19:50:26.185338+0000","last_fullsized":"2024-04-12T19:50:26.185338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:26:54.420503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688867+0000","last_change":"2024-04-12T19:50:23.678658+0000","last_active":"2024-04-12T19:50:25.688867+0000","last_peered":"2024-04-12T19:50:25.688867+0000","last_clean":"2024-04-12T19:50:25.688867+0000","last_became_active":"2024-04-12T19:50:23.676520+0000","last_became_peered":"2024-04-12T19:50:23.676520+0000","last_unstale":"2024-04-12T19:50:25.688867+0000","last_undegraded":"2024-04-12T19:50:25.688867+0000","last_fullsized":"2024-04-12T19:50:25.688867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:43:06.372962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.673494+0000","last_change":"2024-04-12T19:50:24.684296+0000","last_active":"2024-04-12T19:50:25.673494+0000","last_peered":"2024-04-12T19:50:25.673494+0000","last_clean":"2024-04-12T19:50:25.673494+0000","last_became_active":"2024-04-12T19:50:24.683867+0000","last_became_peered":"2024-04-12T19:50:24.683867+0000","last_unstale":"2024-04-12T19:50:25.673494+0000","last_undegraded":"2024-04-12T19:50:25.673494+0000","last_fullsized":"2024-04-12T19:50:25.673494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:29:25.600406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.688543+0000","last_change":"2024-04-12T19:50:23.680894+0000","last_active":"2024-04-12T19:50:25.688543+0000","last_peered":"2024-04-12T19:50:25.688543+0000","last_clean":"2024-04-12T19:50:25.688543+0000","last_became_active":"2024-04-12T19:50:23.679663+0000","last_became_peered":"2024-04-12T19:50:23.679663+0000","last_unstale":"2024-04-12T19:50:25.688543+0000","last_undegraded":"2024-04-12T19:50:25.688543+0000","last_fullsized":"2024-04-12T19:50:25.688543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194138+0000","last_change":"2024-04-12T19:50:24.695055+0000","last_active":"2024-04-12T19:50:26.194138+0000","last_peered":"2024-04-12T19:50:26.194138+0000","last_clean":"2024-04-12T19:50:26.194138+0000","last_became_active":"2024-04-12T19:50:24.694565+0000","last_became_peered":"2024-04-12T19:50:24.694565+0000","last_unstale":"2024-04-12T19:50:26.194138+0000","last_undegraded":"2024-04-12T19:50:26.194138+0000","last_fullsized":"2024-04-12T19:50:26.194138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:23:19.492451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194545+0000","last_change":"2024-04-12T19:50:23.684953+0000","last_active":"2024-04-12T19:50:26.194545+0000","last_peered":"2024-04-12T19:50:26.194545+0000","last_clean":"2024-04-12T19:50:26.194545+0000","last_became_active":"2024-04-12T19:50:23.684555+0000","last_became_peered":"2024-04-12T19:50:23.684555+0000","last_unstale":"2024-04-12T19:50:26.194545+0000","last_undegraded":"2024-04-12T19:50:26.194545+0000","last_fullsized":"2024-04-12T19:50:26.194545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:40:04.310673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:26.194574+0000","last_change":"2024-04-12T19:50:24.701879+0000","last_active":"2024-04-12T19:50:26.194574+0000","last_peered":"2024-04-12T19:50:26.194574+0000","last_clean":"2024-04-12T19:50:26.194574+0000","last_became_active":"2024-04-12T19:50:24.701573+0000","last_became_peered":"2024-04-12T19:50:24.701573+0000","last_unstale":"2024-04-12T19:50:26.194574+0000","last_undegraded":"2024-04-12T19:50:26.194574+0000","last_fullsized":"2024-04-12T19:50:26.194574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:23.632776+0000","last_clean_scrub_stamp":"2024-04-12T19:50:23.632776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:09:17.202771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-12T19:50:25.683114+0000","last_change":"2024-04-12T19:50:23.673049+0000","last_active":"2024-04-12T19:50:25.683114+0000","last_peered":"2024-04-12T19:50:25.683114+0000","last_clean":"2024-04-12T19:50:25.683114+0000","last_became_active":"2024-04-12T19:50:23.672610+0000","last_became_peered":"2024-04-12T19:50:23.672610+0000","last_unstale":"2024-04-12T19:50:25.683114+0000","last_undegraded":"2024-04-12T19:50:25.683114+0000","last_fullsized":"2024-04-12T19:50:25.683114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:22.618310+0000","last_clean_scrub_stamp":"2024-04-12T19:50:22.618310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:08:10.948489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:50:33.554 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-12T19:50:33.554 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-12T19:50:33.554 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-12T19:50:33.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-12T19:50:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.809+0000 7f45c7e00640 1 Processor -- start 2024-04-12T19:50:33.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.809+0000 7f45c7e00640 1 -- start start 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8072660 0x7f45c815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c8162380 0x7f45c80af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c8162940 con 0x7f45c8148130 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f45c8148510 con 0x7f45c8162380 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45c815fd70 con 0x7f45c8072660 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c8162380 0x7f45c80af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45c8162380 0x7f45c80af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33058/0 (socket says 172.21.15.32:33058) 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c7400640 1 -- 172.21.15.32:0/3180318230 learned_addr learned my addr 172.21.15.32:0/3180318230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.811+0000 7f45c6000640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8072660 0x7f45c815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.812+0000 7f45c7400640 1 -- 172.21.15.32:0/3180318230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8072660 msgr2=0x7f45c815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.812+0000 7f45c7400640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8072660 0x7f45c815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.812+0000 7f45c7400640 1 -- 172.21.15.32:0/3180318230 >> v1:172.21.15.169:6789/0 conn(0x7f45c8148130 legacy=0x7f45c8071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:33.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.812+0000 7f45c7400640 1 -- 172.21.15.32:0/3180318230 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c8162c20 con 0x7f45c8162380 2024-04-12T19:50:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.813+0000 7f45c7400640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c80af1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45c8162810 tx=0x7f45c002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f436d7f3285f91e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.813+0000 7f45c5600640 1 -- 172.21.15.32:0/3180318230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c002ca30 con 0x7f45c8162380 2024-04-12T19:50:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.813+0000 7f45c5600640 1 -- 172.21.15.32:0/3180318230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c002cbd0 con 0x7f45c8162380 2024-04-12T19:50:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.813+0000 7f45c5600640 1 -- 172.21.15.32:0/3180318230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00343f0 con 0x7f45c8162380 2024-04-12T19:50:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 -- 172.21.15.32:0/3180318230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 msgr2=0x7f45c80af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c80af1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f45c8162810 tx=0x7f45c002be90 comp rx=0 tx=0).stop 2024-04-12T19:50:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 -- 172.21.15.32:0/3180318230 shutdown_connections 2024-04-12T19:50:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c80af1b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 --2- 172.21.15.32:0/3180318230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8072660 0x7f45c815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 -- 172.21.15.32:0/3180318230 >> 172.21.15.32:0/3180318230 conn(0x7f45c806b930 msgr2=0x7f45c806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 -- 172.21.15.32:0/3180318230 shutdown_connections 2024-04-12T19:50:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.814+0000 7f45c7e00640 1 -- 172.21.15.32:0/3180318230 wait complete. 2024-04-12T19:50:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.816+0000 7f45c7e00640 1 Processor -- start 2024-04-12T19:50:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.816+0000 7f45c7e00640 1 -- start start 2024-04-12T19:50:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c8072660 0x7f45c81f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8148130 0x7f45c813d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80aeb90 con 0x7f45c8162380 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81631e0 con 0x7f45c8072660 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80aed10 con 0x7f45c8148130 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33060/0 (socket says 172.21.15.32:33060) 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c7400640 1 -- 172.21.15.32:0/2804258378 learned_addr learned my addr 172.21.15.32:0/2804258378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.817+0000 7f45c6a00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c8072660 0x7f45c81f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c6000640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8148130 0x7f45c813d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 -- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8148130 msgr2=0x7f45c813d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8148130 0x7f45c813d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 -- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c8072660 msgr2=0x7f45c81f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c8072660 0x7f45c81f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 -- 172.21.15.32:0/2804258378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c81f4cd0 con 0x7f45c8162380 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7400640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45c0036040 tx=0x7f45c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0043020 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c003e450 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0009510 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c81f4fb0 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.818+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81f4980 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.819+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c80755c0 con 0x7f45c8162380 2024-04-12T19:50:33.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.820+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45c0046020 con 0x7f45c8162380 2024-04-12T19:50:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.821+0000 7f45bea00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 0x7f45a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.821+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f45c00ae470 con 0x7f45c8162380 2024-04-12T19:50:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.821+0000 7f45c6a00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 0x7f45a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.822+0000 7f45c6a00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 0x7f45a406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45b8004ea0 tx=0x7f45b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:33.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:33.826+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45c004d030 con 0x7f45c8162380 2024-04-12T19:50:34.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.094+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f45c80b2050 con 0x7f45c8162380 2024-04-12T19:50:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.095+0000 7f45bea00640 1 -- 172.21.15.32:0/2804258378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f45c007c800 con 0x7f45c8162380 2024-04-12T19:50:34.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:34.095 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.097+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 msgr2=0x7f45a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.097+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 0x7f45a406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45b8004ea0 tx=0x7f45b8009040 comp rx=0 tx=0).stop 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.097+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 msgr2=0x7f45c813dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.097+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45c0036040 tx=0x7f45c0004a40 comp rx=0 tx=0).stop 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 shutdown_connections 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f45a406c8f0 0x7f45a406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c8162380 0x7f45c813dbb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45c8148130 0x7f45c813d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 --2- 172.21.15.32:0/2804258378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c8072660 0x7f45c81f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 >> 172.21.15.32:0/2804258378 conn(0x7f45c806b930 msgr2=0x7f45c80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 shutdown_connections 2024-04-12T19:50:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.098+0000 7f45c7e00640 1 -- 172.21.15.32:0/2804258378 wait complete. 2024-04-12T19:50:34.116 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-12T19:50:34.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.340+0000 7faee9c00640 1 Processor -- start 2024-04-12T19:50:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.340+0000 7faee9c00640 1 -- start start 2024-04-12T19:50:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faee9c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faee4072660 0x7faee406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faee9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faee415cbe0 0x7faee415d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faee9c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee415dbb0 con 0x7faee40cee00 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faee9c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faee406f4f0 con 0x7faee4072660 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faee9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faee415d8d0 con 0x7faee415cbe0 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.342+0000 7faedb400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faee4072660 0x7faee406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faee4072660 0x7faee406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35164/0 (socket says 172.21.15.32:35164) 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 -- 172.21.15.32:0/2916069996 learned_addr learned my addr 172.21.15.32:0/2916069996 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faee9200640 1 --2- 172.21.15.32:0/2916069996 >> v2:172.21.15.169:3300/0 conn(0x7faee415cbe0 0x7faee415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 -- 172.21.15.32:0/2916069996 >> v2:172.21.15.169:3300/0 conn(0x7faee415cbe0 msgr2=0x7faee415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 --2- 172.21.15.32:0/2916069996 >> v2:172.21.15.169:3300/0 conn(0x7faee415cbe0 0x7faee415d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 -- 172.21.15.32:0/2916069996 >> v1:172.21.15.169:6789/0 conn(0x7faee40cee00 legacy=0x7faee4071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.343+0000 7faedb400640 1 -- 172.21.15.32:0/2916069996 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee406dde0 con 0x7faee4072660 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.344+0000 7faedb400640 1 --2- 172.21.15.32:0/2916069996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee4072660 0x7faee406e8c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee40b02a0 tx=0x7faecc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4acd2b489f6175 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.344+0000 7faee3400640 1 -- 172.21.15.32:0/2916069996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc035020 con 0x7faee4072660 2024-04-12T19:50:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.344+0000 7faee3400640 1 -- 172.21.15.32:0/2916069996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc031870 con 0x7faee4072660 2024-04-12T19:50:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.344+0000 7faee3400640 1 -- 172.21.15.32:0/2916069996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc031c40 con 0x7faee4072660 2024-04-12T19:50:34.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 -- 172.21.15.32:0/2916069996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee4072660 msgr2=0x7faee406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 --2- 172.21.15.32:0/2916069996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee4072660 0x7faee406e8c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee40b02a0 tx=0x7faecc02b910 comp rx=0 tx=0).stop 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 -- 172.21.15.32:0/2916069996 shutdown_connections 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 --2- 172.21.15.32:0/2916069996 >> v2:172.21.15.169:3300/0 conn(0x7faee415cbe0 0x7faee415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 --2- 172.21.15.32:0/2916069996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee4072660 0x7faee406e8c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 -- 172.21.15.32:0/2916069996 >> 172.21.15.32:0/2916069996 conn(0x7faee406b930 msgr2=0x7faee406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 -- 172.21.15.32:0/2916069996 shutdown_connections 2024-04-12T19:50:34.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.345+0000 7faee9c00640 1 -- 172.21.15.32:0/2916069996 wait complete. 2024-04-12T19:50:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.347+0000 7faee9c00640 1 Processor -- start 2024-04-12T19:50:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.347+0000 7faee9c00640 1 -- start start 2024-04-12T19:50:34.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 0x7faee4159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 0x7faee41f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faee415d5b0 con 0x7faee40cee00 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faee406e3a0 con 0x7faee4072660 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faee9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faee415dda0 con 0x7faee415cbe0 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faedb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faedb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35174/0 (socket says 172.21.15.32:35174) 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.348+0000 7faedb400640 1 -- 172.21.15.32:0/3348336559 learned_addr learned my addr 172.21.15.32:0/3348336559 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee9200640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 0x7faee41f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee3e00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 0x7faee4159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 -- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 msgr2=0x7faee41f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 0x7faee41f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 -- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 msgr2=0x7faee4159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 0x7faee4159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 -- 172.21.15.32:0/3348336559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee41f0e60 con 0x7faee40cee00 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee9200640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 0x7faee41f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee3e00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 0x7faee4159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faedb400640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faecc000c00 tx=0x7faecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0443f0 con 0x7faee40cee00 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc044590 con 0x7faee40cee00 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc044980 con 0x7faee40cee00 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.349+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee40ab450 con 0x7faee40cee00 2024-04-12T19:50:34.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.350+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee40aba10 con 0x7faee40cee00 2024-04-12T19:50:34.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.351+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee40755c0 con 0x7faee40cee00 2024-04-12T19:50:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.352+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faecc04b060 con 0x7faee40cee00 2024-04-12T19:50:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.352+0000 7faee0c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 0x7faec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.353+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faecc0ade90 con 0x7faee40cee00 2024-04-12T19:50:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.353+0000 7faee3e00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 0x7faec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.353+0000 7faee3e00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 0x7faec006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faee40cde20 tx=0x7faed4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.357+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faecc03f070 con 0x7faee40cee00 2024-04-12T19:50:34.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.562+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faee406cf30 con 0x7faee40cee00 2024-04-12T19:50:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.564+0000 7faee0c00640 1 -- 172.21.15.32:0/3348336559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7faecc07c2a0 con 0x7faee40cee00 2024-04-12T19:50:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:34.565 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:34.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 msgr2=0x7faec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 0x7faec006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faee40cde20 tx=0x7faed4009040 comp rx=0 tx=0).stop 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 msgr2=0x7faee41f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faecc000c00 tx=0x7faecc0046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 shutdown_connections 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faec006c8f0 0x7faec006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee415cbe0 0x7faee41f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faee40cee00 0x7faee41f0290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 --2- 172.21.15.32:0/3348336559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee4072660 0x7faee4159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 >> 172.21.15.32:0/3348336559 conn(0x7faee406b930 msgr2=0x7faee4074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 shutdown_connections 2024-04-12T19:50:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.567+0000 7faee9c00640 1 -- 172.21.15.32:0/3348336559 wait complete. 2024-04-12T19:50:34.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:34.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:34.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.818+0000 7f5f1f000640 1 Processor -- start 2024-04-12T19:50:34.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.818+0000 7f5f1f000640 1 -- start start 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f180cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f180ce070 0x7f5f180b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f180b59f0 0x7f5f1806f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5f18145550 con 0x7f5f180b59f0 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5f180b4410 con 0x7f5f180ce070 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180b46f0 con 0x7f5f18145170 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f180b59f0 0x7f5f1806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f180b59f0 0x7f5f1806f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35188/0 (socket says 172.21.15.32:35188) 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f1d000640 1 -- 172.21.15.32:0/659088517 learned_addr learned my addr 172.21.15.32:0/659088517 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:34.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f17e00640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f180cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.820+0000 7f5f0fe00640 1 --2- 172.21.15.32:0/659088517 >> v2:172.21.15.169:3300/0 conn(0x7f5f180ce070 0x7f5f180b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.821+0000 7f5f1d000640 1 -- 172.21.15.32:0/659088517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 msgr2=0x7f5f180cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.821+0000 7f5f1d000640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f180cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.821+0000 7f5f1d000640 1 -- 172.21.15.32:0/659088517 >> v2:172.21.15.169:3300/0 conn(0x7f5f180ce070 msgr2=0x7f5f180b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.821+0000 7f5f1d000640 1 --2- 172.21.15.32:0/659088517 >> v2:172.21.15.169:3300/0 conn(0x7f5f180ce070 0x7f5f180b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.821+0000 7f5f1d000640 1 -- 172.21.15.32:0/659088517 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f180b49d0 con 0x7f5f180b59f0 2024-04-12T19:50:34.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.822+0000 7f5f1d000640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1806f260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5f1806c9d0 tx=0x7f5f1002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912ea996d847ab1d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.822+0000 7f5f17400640 1 -- 172.21.15.32:0/659088517 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1002ca70 con 0x7f5f180b59f0 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.822+0000 7f5f17400640 1 -- 172.21.15.32:0/659088517 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1002cc10 con 0x7f5f180b59f0 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.822+0000 7f5f17400640 1 -- 172.21.15.32:0/659088517 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100343f0 con 0x7f5f180b59f0 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 -- 172.21.15.32:0/659088517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 msgr2=0x7f5f1806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1806f260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5f1806c9d0 tx=0x7f5f1002bed0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 -- 172.21.15.32:0/659088517 shutdown_connections 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1806f260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 --2- 172.21.15.32:0/659088517 >> v2:172.21.15.169:3300/0 conn(0x7f5f180ce070 0x7f5f180b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 --2- 172.21.15.32:0/659088517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f180cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 -- 172.21.15.32:0/659088517 >> 172.21.15.32:0/659088517 conn(0x7f5f180b2460 msgr2=0x7f5f180b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 -- 172.21.15.32:0/659088517 shutdown_connections 2024-04-12T19:50:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.823+0000 7f5f1f000640 1 -- 172.21.15.32:0/659088517 wait complete. 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.825+0000 7f5f1f000640 1 Processor -- start 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.825+0000 7f5f1f000640 1 -- start start 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 0x7f5f181f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f1813aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180b4030 con 0x7f5f180b59f0 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180b4f90 con 0x7f5f180ce070 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180b41b0 con 0x7f5f18145170 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35190/0 (socket says 172.21.15.32:35190) 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f17e00640 1 -- 172.21.15.32:0/3670478423 learned_addr learned my addr 172.21.15.32:0/3670478423 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.826+0000 7f5f1d000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f1813aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 -- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 msgr2=0x7f5f1813aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f0fe00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 0x7f5f181f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f1813aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 -- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 msgr2=0x7f5f181f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 0x7f5f181f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 -- 172.21.15.32:0/3670478423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1813b410 con 0x7f5f180b59f0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f0fe00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 0x7f5f181f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f17e00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d5f0 tx=0x7f5f0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f080125f0 con 0x7f5f180b59f0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f08019070 con 0x7f5f180b59f0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f080129d0 con 0x7f5f180b59f0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1813b6f0 con 0x7f5f180b59f0 2024-04-12T19:50:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.827+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f18134c30 con 0x7f5f180b59f0 2024-04-12T19:50:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.829+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1813a480 con 0x7f5f180b59f0 2024-04-12T19:50:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.829+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f08012b70 con 0x7f5f180b59f0 2024-04-12T19:50:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.831+0000 7f5f14c00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.831+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5f0808bf00 con 0x7f5f180b59f0 2024-04-12T19:50:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.831+0000 7f5f0fe00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.832+0000 7f5f0fe00640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f1813e1d0 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:34.836+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f1813a480 con 0x7f5f180b59f0 2024-04-12T19:50:35.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.038+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f1813bc10 con 0x7f5f180b59f0 2024-04-12T19:50:35.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.039+0000 7f5f14c00640 1 -- 172.21.15.32:0/3670478423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f5f0805a320 con 0x7f5f180b59f0 2024-04-12T19:50:35.039 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 msgr2=0x7f5ef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f1813e1d0 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2024-04-12T19:50:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 msgr2=0x7f5f1813dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d5f0 tx=0x7f5f0800dab0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 shutdown_connections 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f18145170 0x7f5f1813aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f180ce070 0x7f5f181f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 --2- 172.21.15.32:0/3670478423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f180b59f0 0x7f5f1813dc90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 >> 172.21.15.32:0/3670478423 conn(0x7f5f180b2460 msgr2=0x7f5f18072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 shutdown_connections 2024-04-12T19:50:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.042+0000 7f5f1f000640 1 -- 172.21.15.32:0/3670478423 wait complete. 2024-04-12T19:50:35.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:50:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.278+0000 7fa1ff200640 1 Processor -- start 2024-04-12T19:50:35.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.278+0000 7fa1ff200640 1 -- start start 2024-04-12T19:50:35.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1ff200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2000725f0 0x7fa2000ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa20006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1ff200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2000cd2d0 con 0x7fa2001461c0 2024-04-12T19:50:35.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1ff200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2000cf1d0 con 0x7fa2000725f0 2024-04-12T19:50:35.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2000cd080 con 0x7fa2000ce9e0 2024-04-12T19:50:35.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1fde00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2001461c0 0x7fa200071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51930/0 (socket says 172.21.15.32:51930) 2024-04-12T19:50:35.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.280+0000 7fa1fde00640 1 -- 172.21.15.32:0/3388296688 learned_addr learned my addr 172.21.15.32:0/3388296688 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:35.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fe800640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa20006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1f7e00640 1 --2- 172.21.15.32:0/3388296688 >> v2:172.21.15.169:3300/0 conn(0x7fa2000725f0 0x7fa2000ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542983564 0 0) 0x7fa2000cd2d0 con 0x7fa2001461c0 2024-04-12T19:50:35.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e4003520 con 0x7fa2001461c0 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1091651667 0 0) 0x7fa1e4003520 con 0x7fa2001461c0 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 msgr2=0x7fa20006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa20006df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 >> v2:172.21.15.169:3300/0 conn(0x7fa2000725f0 msgr2=0x7fa2000ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 --2- 172.21.15.32:0/3388296688 >> v2:172.21.15.169:3300/0 conn(0x7fa2000725f0 0x7fa2000ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2000cd5b0 con 0x7fa2001461c0 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fa1f0001670 con 0x7fa2001461c0 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 >> v1:172.21.15.32:6789/0 conn(0x7fa2001461c0 legacy=0x7fa200071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa1e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4003f30 0x7fa1e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e4004d40 0x7fa1e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000cd080 con 0x7fa1e4004d40 2024-04-12T19:50:35.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400b9c0 con 0x7fa1e4003f30 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.281+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e400bb40 con 0x7fa2000ce9e0 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e4004d40 0x7fa1e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1f7e00640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4003f30 0x7fa1e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fde00640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa1e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 -- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 msgr2=0x7fa1e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa1e40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 -- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4003f30 msgr2=0x7fa1e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4003f30 0x7fa1e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fe800640 1 -- 172.21.15.32:0/3388296688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e4003520 con 0x7fa1e4004d40 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.282+0000 7fa1fde00640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000ce9e0 0x7fa1e40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.283+0000 7fa1fe800640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e4004d40 0x7fa1e40051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009fc0 tx=0x7fa1e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d10802dd869044 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.283+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8039070 con 0x7fa1e4004d40 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.283+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f802cc00 con 0x7fa1e4004d40 2024-04-12T19:50:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.283+0000 7fa1fd400640 1 -- 172.21.15.32:0/3388296688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8009c10 con 0x7fa1e4004d40 2024-04-12T19:50:35.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 -- 172.21.15.32:0/3388296688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e4004d40 msgr2=0x7fa1e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 --2- 172.21.15.32:0/3388296688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e4004d40 0x7fa1e40051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009fc0 tx=0x7fa1e4005830 comp rx=0 tx=0).stop 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1fe800640 1 -- 172.21.15.32:0/3388296688 reap_dead start 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 -- 172.21.15.32:0/3388296688 shutdown_connections 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 -- 172.21.15.32:0/3388296688 >> 172.21.15.32:0/3388296688 conn(0x7fa20006b930 msgr2=0x7fa20006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 -- 172.21.15.32:0/3388296688 shutdown_connections 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.284+0000 7fa1ff200640 1 -- 172.21.15.32:0/3388296688 wait complete. 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.286+0000 7fa1ff200640 1 Processor -- start 2024-04-12T19:50:35.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.286+0000 7fa1ff200640 1 -- start start 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 0x7fa20013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000b13c0 0x7fa2000b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000cee70 con 0x7fa20015b390 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000cdb70 con 0x7fa1e4004d40 2024-04-12T19:50:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.287+0000 7fa1ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa20006df20 con 0x7fa2000b13c0 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35200/0 (socket says 172.21.15.32:35200) 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 -- 172.21.15.32:0/2924417487 learned_addr learned my addr 172.21.15.32:0/2924417487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1f7e00640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000b13c0 0x7fa2000b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fde00640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 0x7fa20013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 -- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000b13c0 msgr2=0x7fa2000b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000b13c0 0x7fa2000b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 -- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 msgr2=0x7fa20013de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 0x7fa20013de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 -- 172.21.15.32:0/2924417487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20015c330 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fde00640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 0x7fa20013de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.288+0000 7fa1fe800640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8037040 tx=0x7fa1f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.289+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8039030 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.289+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f803a040 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.289+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8009c10 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.289+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa200157400 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.289+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001576a0 con 0x7fa20015b390 2024-04-12T19:50:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.290+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2000755c0 con 0x7fa20015b390 2024-04-12T19:50:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.291+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1f8045030 con 0x7fa20015b390 2024-04-12T19:50:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.292+0000 7fa1f6000640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 0x7fa1dc06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.292+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f80ad840 con 0x7fa20015b390 2024-04-12T19:50:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.292+0000 7fa1fde00640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 0x7fa1dc06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.293+0000 7fa1fde00640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 0x7fa1dc06ed50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0001eb0 tx=0x7fa1f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.297+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1f804c060 con 0x7fa20015b390 2024-04-12T19:50:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.499+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa200072e40 con 0x7fa20015b390 2024-04-12T19:50:35.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.500+0000 7fa1f6000640 1 -- 172.21.15.32:0/2924417487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fa1f807bc60 con 0x7fa20015b390 2024-04-12T19:50:35.500 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:50:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 msgr2=0x7fa1dc06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 0x7fa1dc06ed50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0001eb0 tx=0x7fa1f0009040 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 msgr2=0x7fa20015b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8037040 tx=0x7fa1f8009750 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 shutdown_connections 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa1dc06c890 0x7fa1dc06ed50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa20015b390 0x7fa20015b810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2000b13c0 0x7fa2000b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 --2- 172.21.15.32:0/2924417487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e4004d40 0x7fa20013de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 >> 172.21.15.32:0/2924417487 conn(0x7fa20006b930 msgr2=0x7fa20006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 shutdown_connections 2024-04-12T19:50:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.502+0000 7fa1ff200640 1 -- 172.21.15.32:0/2924417487 wait complete. 2024-04-12T19:50:35.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:50:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.744+0000 7ffa91000640 1 Processor -- start 2024-04-12T19:50:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.745+0000 7ffa91000640 1 -- start start 2024-04-12T19:50:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa91000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa8c1461c0 0x7ffa8c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa91000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c072420 0x7ffa8c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa91000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8c0cd2d0 con 0x7ffa8c0ce9e0 2024-04-12T19:50:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa91000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffa8c0cf1d0 con 0x7ffa8c1461c0 2024-04-12T19:50:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa91000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c072880 con 0x7ffa8c072420 2024-04-12T19:50:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa8be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa8c0ce9e0 0x7ffa8c160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51944/0 (socket says 172.21.15.32:51944) 2024-04-12T19:50:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.746+0000 7ffa8be00640 1 -- 172.21.15.32:0/1367169236 learned_addr learned my addr 172.21.15.32:0/1367169236 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.747+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1367169236 >> v2:172.21.15.169:3300/0 conn(0x7ffa8c1461c0 0x7ffa8c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.747+0000 7ffa83e00640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c072420 0x7ffa8c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.747+0000 7ffa8aa00640 1 -- 172.21.15.32:0/1367169236 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815531749 0 0) 0x7ffa8c0cd2d0 con 0x7ffa8c0ce9e0 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.747+0000 7ffa8aa00640 1 -- 172.21.15.32:0/1367169236 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa70003520 con 0x7ffa8c0ce9e0 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.748+0000 7ffa8b400640 1 -- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c072420 msgr2=0x7ffa8c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.748+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c072420 0x7ffa8c0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.748+0000 7ffa8b400640 1 -- 172.21.15.32:0/1367169236 >> v1:172.21.15.32:6789/0 conn(0x7ffa8c0ce9e0 legacy=0x7ffa8c160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.748+0000 7ffa8b400640 1 -- 172.21.15.32:0/1367169236 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c0cd5b0 con 0x7ffa8c1461c0 2024-04-12T19:50:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.749+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c1461c0 0x7ffa8c071ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0097a0 tx=0x7ffa7c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5becea544e665ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.753+0000 7ffa8aa00640 1 -- 172.21.15.32:0/1367169236 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c035020 con 0x7ffa8c1461c0 2024-04-12T19:50:35.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.753+0000 7ffa8aa00640 1 -- 172.21.15.32:0/1367169236 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c031830 con 0x7ffa8c1461c0 2024-04-12T19:50:35.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.753+0000 7ffa8aa00640 1 -- 172.21.15.32:0/1367169236 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c031c00 con 0x7ffa8c1461c0 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.753+0000 7ffa91000640 1 -- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c1461c0 msgr2=0x7ffa8c071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.753+0000 7ffa91000640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c1461c0 0x7ffa8c071ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0097a0 tx=0x7ffa7c02b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 -- 172.21.15.32:0/1367169236 shutdown_connections 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c072420 0x7ffa8c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 --2- 172.21.15.32:0/1367169236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c1461c0 0x7ffa8c071ee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 -- 172.21.15.32:0/1367169236 >> 172.21.15.32:0/1367169236 conn(0x7ffa8c06b930 msgr2=0x7ffa8c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 -- 172.21.15.32:0/1367169236 shutdown_connections 2024-04-12T19:50:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.754+0000 7ffa91000640 1 -- 172.21.15.32:0/1367169236 wait complete. 2024-04-12T19:50:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.756+0000 7ffa91000640 1 Processor -- start 2024-04-12T19:50:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.756+0000 7ffa91000640 1 -- start start 2024-04-12T19:50:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c0ce9e0 0x7ffa8c143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c1461c0 0x7ffa8c13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c0cdb70 con 0x7ffa8c072420 2024-04-12T19:50:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c0cd080 con 0x7ffa8c0ce9e0 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa91000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c0cee70 con 0x7ffa8c1461c0 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa8b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa8b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35204/0 (socket says 172.21.15.32:35204) 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa8b400640 1 -- 172.21.15.32:0/1184781290 learned_addr learned my addr 172.21.15.32:0/1184781290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa83e00640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c0ce9e0 0x7ffa8c143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.757+0000 7ffa8be00640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c1461c0 0x7ffa8c13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 -- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c1461c0 msgr2=0x7ffa8c13aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c1461c0 0x7ffa8c13aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 -- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c0ce9e0 msgr2=0x7ffa8c143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c0ce9e0 0x7ffa8c143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 -- 172.21.15.32:0/1184781290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c144790 con 0x7ffa8c072420 2024-04-12T19:50:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa8b400640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098c0 tx=0x7ffa7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c040070 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c03e070 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.758+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c035040 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.759+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c144a70 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.759+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c145030 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.760+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8c0755c0 con 0x7ffa8c072420 2024-04-12T19:50:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.761+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa7c038020 con 0x7ffa8c072420 2024-04-12T19:50:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.761+0000 7ffa83400640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 0x7ffa6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.762+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7c0ad080 con 0x7ffa8c072420 2024-04-12T19:50:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.762+0000 7ffa83e00640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 0x7ffa6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.762+0000 7ffa83e00640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 0x7ffa6806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c1359d0 tx=0x7ffa74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:35.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.766+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa7c07b4a0 con 0x7ffa8c072420 2024-04-12T19:50:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.970+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa8c072e40 con 0x7ffa8c072420 2024-04-12T19:50:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.971+0000 7ffa83400640 1 -- 172.21.15.32:0/1184781290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ffa7c07ada0 con 0x7ffa8c072420 2024-04-12T19:50:35.971 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:50:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 msgr2=0x7ffa6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 0x7ffa6806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c1359d0 tx=0x7ffa74008040 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 msgr2=0x7ffa8c1f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0098c0 tx=0x7ffa7c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 shutdown_connections 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ffa6806c8f0 0x7ffa6806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa8c1461c0 0x7ffa8c13aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa8c0ce9e0 0x7ffa8c143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 --2- 172.21.15.32:0/1184781290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa8c072420 0x7ffa8c1f4070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 >> 172.21.15.32:0/1184781290 conn(0x7ffa8c06b930 msgr2=0x7ffa8c141b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 shutdown_connections 2024-04-12T19:50:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:35.974+0000 7ffa91000640 1 -- 172.21.15.32:0/1184781290 wait complete. 2024-04-12T19:50:35.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:50:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.217+0000 7f2f91a00640 1 Processor -- start 2024-04-12T19:50:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.218+0000 7f2f91a00640 1 -- start start 2024-04-12T19:50:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.219+0000 7f2f91a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f8c162020 0x7f2f8c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.219+0000 7f2f91a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c0d0090 con 0x7f2f8c0cf110 2024-04-12T19:50:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.219+0000 7f2f91a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c0cfad0 con 0x7f2f8c0d1c90 2024-04-12T19:50:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.219+0000 7f2f91a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8c163440 con 0x7f2f8c162020 2024-04-12T19:50:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.220+0000 7f2f83400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f8c0cf110 0x7f2f8c0d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51950/0 (socket says 172.21.15.32:51950) 2024-04-12T19:50:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.220+0000 7f2f83400640 1 -- 172.21.15.32:0/1730747373 learned_addr learned my addr 172.21.15.32:0/1730747373 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.220+0000 7f2f8be00640 1 --2- 172.21.15.32:0/1730747373 >> v2:172.21.15.169:3300/0 conn(0x7f2f8c162020 0x7f2f8c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.220+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147628350 0 0) 0x7f2f74001110 con 0x7f2f8c0cf110 2024-04-12T19:50:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.220+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f70003520 con 0x7f2f8c0cf110 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868379584 0 0) 0x7f2f70003520 con 0x7f2f8c0cf110 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 >> v2:172.21.15.169:3300/0 conn(0x7f2f8c162020 msgr2=0x7f2f8c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 --2- 172.21.15.32:0/1730747373 >> v2:172.21.15.169:3300/0 conn(0x7f2f8c162020 0x7f2f8c162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 >> v1:172.21.15.169:6789/0 conn(0x7f2f8c0d1c90 legacy=0x7f2f8c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c0cfdb0 con 0x7f2f8c0cf110 2024-04-12T19:50:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f2f74001840 con 0x7f2f8c0cf110 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 >> v1:172.21.15.32:6789/0 conn(0x7f2f8c0cf110 legacy=0x7f2f8c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 0x7f2f70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f70004800 0x7f2f70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700051c0 0x7f2f7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c163440 con 0x7f2f700039b0 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7000c060 con 0x7f2f700051c0 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7000c260 con 0x7f2f70004800 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.221+0000 7f2f8be00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 0x7f2f70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f83400640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f70004800 0x7f2f70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f91000640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700051c0 0x7f2f7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f8be00640 1 -- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f70004800 msgr2=0x7f2f70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f8be00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f70004800 0x7f2f70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f8be00640 1 -- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700051c0 msgr2=0x7f2f7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f8be00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700051c0 0x7f2f7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f8be00640 1 -- 172.21.15.32:0/1730747373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f70003520 con 0x7f2f700039b0 2024-04-12T19:50:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f83400640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f70004800 0x7f2f70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f91000640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700051c0 0x7f2f7000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.222+0000 7f2f91000640 1 -- 172.21.15.32:0/1730747373 reap_dead start 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.223+0000 7f2f8be00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 0x7f2f70003dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009ac0 tx=0x7f2f70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c701558d378657 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.223+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c039070 con 0x7f2f700039b0 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.223+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c02c6d0 con 0x7f2f700039b0 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.223+0000 7f2f8b400640 1 -- 172.21.15.32:0/1730747373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c009690 con 0x7f2f700039b0 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 -- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 msgr2=0x7f2f70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 0x7f2f70003dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009ac0 tx=0x7f2f70005830 comp rx=0 tx=0).stop 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 -- 172.21.15.32:0/1730747373 shutdown_connections 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 --2- 172.21.15.32:0/1730747373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f700039b0 0x7f2f70003dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 -- 172.21.15.32:0/1730747373 >> 172.21.15.32:0/1730747373 conn(0x7f2f8c06cf00 msgr2=0x7f2f8c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 -- 172.21.15.32:0/1730747373 shutdown_connections 2024-04-12T19:50:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.224+0000 7f2f91a00640 1 -- 172.21.15.32:0/1730747373 wait complete. 2024-04-12T19:50:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.226+0000 7f2f91a00640 1 Processor -- start 2024-04-12T19:50:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.226+0000 7f2f91a00640 1 -- start start 2024-04-12T19:50:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700039b0 0x7f2f8c14bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f8c13c250 0x7f2f8c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0cf880 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0b7b30 con 0x7f2f700039b0 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.227+0000 7f2f91a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8c0d2e70 con 0x7f2f8c13c250 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35226/0 (socket says 172.21.15.32:35226) 2024-04-12T19:50:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 -- 172.21.15.32:0/2463284840 learned_addr learned my addr 172.21.15.32:0/2463284840 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f8be00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700039b0 0x7f2f8c14bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f91000640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f8c13c250 0x7f2f8c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 -- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f8c13c250 msgr2=0x7f2f8c153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f8c13c250 0x7f2f8c153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 -- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700039b0 msgr2=0x7f2f8c14bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700039b0 0x7f2f8c14bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 -- 172.21.15.32:0/2463284840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c14c050 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.228+0000 7f2f83400640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f740013d0 tx=0x7f2f7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.229+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74019070 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.229+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f740072d0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.229+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f740047e0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.229+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c0c9300 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.229+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c0c98c0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.230+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c0755c0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.231+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7401c070 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.232+0000 7f2f88c00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 0x7f2f6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.232+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7408b4a0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.232+0000 7f2f8be00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 0x7f2f6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.233+0000 7f2f8be00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 0x7f2f6806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c036000 tx=0x7f2f7c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.237+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f8c0d0090 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.481+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f8c0b7650 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.482+0000 7f2f88c00640 1 -- 172.21.15.32:0/2463284840 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f2f740591e0 con 0x7f2f8c0c8d60 2024-04-12T19:50:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:36.483 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:50:36.483 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:50:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 msgr2=0x7f2f6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 0x7f2f6806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c036000 tx=0x7f2f7c030000 comp rx=0 tx=0).stop 2024-04-12T19:50:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 msgr2=0x7f2f8c15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f740013d0 tx=0x7f2f7400dc00 comp rx=0 tx=0).stop 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 shutdown_connections 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f6806c8f0 0x7f2f6806edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f8c13c250 0x7f2f8c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f8c0c8d60 0x7f2f8c15ebd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 --2- 172.21.15.32:0/2463284840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f700039b0 0x7f2f8c14bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 >> 172.21.15.32:0/2463284840 conn(0x7f2f8c06cf00 msgr2=0x7f2f8c0b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:36.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.485+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 shutdown_connections 2024-04-12T19:50:36.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.486+0000 7f2f91a00640 1 -- 172.21.15.32:0/2463284840 wait complete. 2024-04-12T19:50:36.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:36.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.732+0000 7fc3f2c00640 1 Processor -- start 2024-04-12T19:50:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.732+0000 7fc3f2c00640 1 -- start start 2024-04-12T19:50:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.734+0000 7fc3f2c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0cdd50 0x7fc3ec06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.734+0000 7fc3f2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.734+0000 7fc3f2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3ec0ce180 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.734+0000 7fc3f2c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ec0cfe30 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.734+0000 7fc3f2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec0d0110 con 0x7fc3ec06be70 2024-04-12T19:50:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0cdd50 0x7fc3ec06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3ec0cdd50 0x7fc3ec06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35238/0 (socket says 172.21.15.32:35238) 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f1800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc3ec15e8e0 0x7fc3ec0712a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:37384/0 (socket says 172.21.15.32:37384) 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 -- 172.21.15.32:0/2540328219 learned_addr learned my addr 172.21.15.32:0/2540328219 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 -- 172.21.15.32:0/2540328219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 msgr2=0x7fc3ec0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 --2- 172.21.15.32:0/2540328219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 -- 172.21.15.32:0/2540328219 >> v1:172.21.15.169:6789/0 conn(0x7fc3ec15e8e0 legacy=0x7fc3ec0712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.735+0000 7fc3f0e00640 1 -- 172.21.15.32:0/2540328219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec0d03f0 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.737+0000 7fc3f0e00640 1 --2- 172.21.15.32:0/2540328219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec06b930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6a7d47f4e4d015 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.737+0000 7fc3ebe00640 1 -- 172.21.15.32:0/2540328219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8034070 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.737+0000 7fc3ebe00640 1 -- 172.21.15.32:0/2540328219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d8030840 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.737+0000 7fc3ebe00640 1 -- 172.21.15.32:0/2540328219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8030c10 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 -- 172.21.15.32:0/2540328219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0cdd50 msgr2=0x7fc3ec06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/2540328219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec06b930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d802b980 comp rx=0 tx=0).stop 2024-04-12T19:50:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 -- 172.21.15.32:0/2540328219 shutdown_connections 2024-04-12T19:50:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/2540328219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/2540328219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec06b930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 -- 172.21.15.32:0/2540328219 >> 172.21.15.32:0/2540328219 conn(0x7fc3ec0b3370 msgr2=0x7fc3ec0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 -- 172.21.15.32:0/2540328219 shutdown_connections 2024-04-12T19:50:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.738+0000 7fc3f2c00640 1 -- 172.21.15.32:0/2540328219 wait complete. 2024-04-12T19:50:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.740+0000 7fc3f2c00640 1 Processor -- start 2024-04-12T19:50:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.740+0000 7fc3f2c00640 1 -- start start 2024-04-12T19:50:36.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0b5110 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0b4f90 con 0x7fc3ec0cdd50 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.741+0000 7fc3f2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec0b5290 con 0x7fc3ec06be70 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35246/0 (socket says 172.21.15.32:35246) 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 -- 172.21.15.32:0/3908754006 learned_addr learned my addr 172.21.15.32:0/3908754006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f1800640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f0e00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 -- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 msgr2=0x7fc3ec0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 -- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3ec0cdd50 msgr2=0x7fc3ec150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.742+0000 7fc3f2200640 1 -- 172.21.15.32:0/3908754006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec161270 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3f2200640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400efc0 tx=0x7fc3e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4018020 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e4014480 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4014870 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec161550 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.743+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec161b10 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.744+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec1538d0 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.745+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3e4005d10 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.746+0000 7fc3e9600640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 0x7fc3cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.746+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc3e408c750 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.746+0000 7fc3f1800640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 0x7fc3cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:36.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.747+0000 7fc3f1800640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 0x7fc3cc06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0000f60 tx=0x7fc3e0007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:36.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.751+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3ec1538d0 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.955+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ec151530 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.959+0000 7fc3e9600640 1 -- 172.21.15.32:0/3908754006 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7fc3e4022070 con 0x7fc3ec15e8e0 2024-04-12T19:50:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:36.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 msgr2=0x7fc3cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 0x7fc3cc06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0000f60 tx=0x7fc3e0007c80 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 msgr2=0x7fc3ec1f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400efc0 tx=0x7fc3e400c490 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 shutdown_connections 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc3cc06c8f0 0x7fc3cc06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.961+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec15e8e0 0x7fc3ec1f4530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.962+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3ec0cdd50 0x7fc3ec150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.962+0000 7fc3f2c00640 1 --2- 172.21.15.32:0/3908754006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3ec06be70 0x7fc3ec0cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.962+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 >> 172.21.15.32:0/3908754006 conn(0x7fc3ec0b3370 msgr2=0x7fc3ec0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:36.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.962+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 shutdown_connections 2024-04-12T19:50:36.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:36.962+0000 7fc3f2c00640 1 -- 172.21.15.32:0/3908754006 wait complete. 2024-04-12T19:50:36.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:36.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:37.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.202+0000 7fa845800640 1 Processor -- start 2024-04-12T19:50:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.202+0000 7fa845800640 1 -- start start 2024-04-12T19:50:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.204+0000 7fa845800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa8400b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.204+0000 7fa845800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa840071370 con 0x7fa8400b7170 2024-04-12T19:50:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.204+0000 7fa845800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8400b49c0 con 0x7fa840148300 2024-04-12T19:50:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.204+0000 7fa845800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa840070aa0 con 0x7fa840070f30 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8400b7170 0x7fa8400ae6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51954/0 (socket says 172.21.15.32:51954) 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83f400640 1 -- 172.21.15.32:0/1201344827 learned_addr learned my addr 172.21.15.32:0/1201344827 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa837e00640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa8400b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882627895 0 0) 0x7fa838001490 con 0x7fa8400b7170 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa824003520 con 0x7fa8400b7170 2024-04-12T19:50:37.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2239608704 0 0) 0x7fa824003520 con 0x7fa8400b7170 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 msgr2=0x7fa8400b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa8400b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 >> v1:172.21.15.169:6789/0 conn(0x7fa840148300 legacy=0x7fa84006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84006fe90 con 0x7fa8400b7170 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.205+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fa838001bc0 con 0x7fa8400b7170 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 >> v1:172.21.15.32:6789/0 conn(0x7fa8400b7170 legacy=0x7fa8400ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8240039e0 0x7fa824003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824004d40 0x7fa8240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa840070aa0 con 0x7fa824004d40 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa82400baa0 con 0x7fa8240039e0 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa82400bca0 con 0x7fa840070f30 2024-04-12T19:50:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824004d40 0x7fa8240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa837e00640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83ea00640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8240039e0 0x7fa824003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 -- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 msgr2=0x7fa824004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840070f30 0x7fa824004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 -- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8240039e0 msgr2=0x7fa824003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8240039e0 0x7fa824003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.206+0000 7fa83f400640 1 -- 172.21.15.32:0/1201344827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa824003520 con 0x7fa824004d40 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa83f400640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824004d40 0x7fa8240051c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa838002230 tx=0x7fa824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d5f836a1dea179 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83800a5a0 con 0x7fa824004d40 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838034650 con 0x7fa824004d40 2024-04-12T19:50:37.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa83e000640 1 -- 172.21.15.32:0/1201344827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838034a40 con 0x7fa824004d40 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa845800640 1 -- 172.21.15.32:0/1201344827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824004d40 msgr2=0x7fa8240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.208+0000 7fa845800640 1 --2- 172.21.15.32:0/1201344827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa824004d40 0x7fa8240051c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa838002230 tx=0x7fa824005830 comp rx=0 tx=0).stop 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.209+0000 7fa83f400640 1 -- 172.21.15.32:0/1201344827 reap_dead start 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.209+0000 7fa845800640 1 -- 172.21.15.32:0/1201344827 shutdown_connections 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.209+0000 7fa845800640 1 -- 172.21.15.32:0/1201344827 >> 172.21.15.32:0/1201344827 conn(0x7fa84009aa40 msgr2=0x7fa84009ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.209+0000 7fa845800640 1 -- 172.21.15.32:0/1201344827 shutdown_connections 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.209+0000 7fa845800640 1 -- 172.21.15.32:0/1201344827 wait complete. 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.210+0000 7fa845800640 1 Processor -- start 2024-04-12T19:50:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.211+0000 7fa845800640 1 -- start start 2024-04-12T19:50:37.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.211+0000 7fa845800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84015e110 0x7fa8401f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.211+0000 7fa845800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840160920 0x7fa8401f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa845800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa845800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa840070730 con 0x7fa8400ccc60 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa845800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa840070450 con 0x7fa84015e110 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa845800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa84006fa70 con 0x7fa840160920 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35272/0 (socket says 172.21.15.32:35272) 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 -- 172.21.15.32:0/1634262779 learned_addr learned my addr 172.21.15.32:0/1634262779 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa837e00640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840160920 0x7fa8401f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83ea00640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84015e110 0x7fa8401f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 -- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840160920 msgr2=0x7fa8401f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840160920 0x7fa8401f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 -- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84015e110 msgr2=0x7fa8401f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84015e110 0x7fa8401f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.212+0000 7fa83f400640 1 -- 172.21.15.32:0/1634262779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8401f9700 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa83f400640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa83802cd40 tx=0x7fa838001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838045070 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83803a040 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83800a580 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8401f57d0 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.213+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401f5d90 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.214+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84006f890 con 0x7fa8400ccc60 2024-04-12T19:50:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.215+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa838048070 con 0x7fa8400ccc60 2024-04-12T19:50:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.216+0000 7fa836a00640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 0x7fa81c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.216+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa8380ad620 con 0x7fa8400ccc60 2024-04-12T19:50:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.217+0000 7fa83ea00640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 0x7fa81c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.217+0000 7fa83ea00640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 0x7fa81c06ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa840160480 tx=0x7fa830009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.221+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa84006f890 con 0x7fa8400ccc60 2024-04-12T19:50:37.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.427+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa840068c80 con 0x7fa8400ccc60 2024-04-12T19:50:37.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.428+0000 7fa836a00640 1 -- 172.21.15.32:0/1634262779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fa8401f9700 con 0x7fa8400ccc60 2024-04-12T19:50:37.428 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 msgr2=0x7fa81c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 0x7fa81c06ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa840160480 tx=0x7fa830009040 comp rx=0 tx=0).stop 2024-04-12T19:50:37.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 msgr2=0x7fa8400cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa83802cd40 tx=0x7fa838001ec0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 shutdown_connections 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa81c06c920 0x7fa81c06ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8400ccc60 0x7fa8400cd0e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa840160920 0x7fa8401f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 --2- 172.21.15.32:0/1634262779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa84015e110 0x7fa8401f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 >> 172.21.15.32:0/1634262779 conn(0x7fa84009aa40 msgr2=0x7fa840141190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 shutdown_connections 2024-04-12T19:50:37.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.431+0000 7fa845800640 1 -- 172.21.15.32:0/1634262779 wait complete. 2024-04-12T19:50:37.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:50:37.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.677+0000 7f22ab200640 1 Processor -- start 2024-04-12T19:50:37.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.678+0000 7f22ab200640 1 -- start start 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22ac0d1570 0x7f22ac0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac15ed30 0x7f22ac0ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22ac0d40a0 0x7f22ac0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f22ac0ba670 con 0x7f22ac0d40a0 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f22ac0cff10 con 0x7f22ac0d1570 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0d1e90 con 0x7f22ac15ed30 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22aa800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22ac0d40a0 0x7f22ac0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22aa800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22ac0d40a0 0x7f22ac0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35284/0 (socket says 172.21.15.32:35284) 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22aa800640 1 -- 172.21.15.32:0/3110599179 learned_addr learned my addr 172.21.15.32:0/3110599179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.679+0000 7f22a9e00640 1 --2- 172.21.15.32:0/3110599179 >> v2:172.21.15.169:3300/0 conn(0x7f22ac0d1570 0x7f22ac0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22a9400640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac15ed30 0x7f22ac0ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22aa800640 1 -- 172.21.15.32:0/3110599179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac15ed30 msgr2=0x7f22ac0ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22aa800640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac15ed30 0x7f22ac0ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22aa800640 1 -- 172.21.15.32:0/3110599179 >> v2:172.21.15.169:3300/0 conn(0x7f22ac0d1570 msgr2=0x7f22ac0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22aa800640 1 --2- 172.21.15.32:0/3110599179 >> v2:172.21.15.169:3300/0 conn(0x7f22ac0d1570 0x7f22ac0cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.680+0000 7f22aa800640 1 -- 172.21.15.32:0/3110599179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac0cfc30 con 0x7f22ac0d40a0 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.681+0000 7f22aa800640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac0d1950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e040 tx=0x7f22a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cdae6b23b8864a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.681+0000 7f22a8a00640 1 -- 172.21.15.32:0/3110599179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a402ca70 con 0x7f22ac0d40a0 2024-04-12T19:50:37.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.681+0000 7f22a8a00640 1 -- 172.21.15.32:0/3110599179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a402cc10 con 0x7f22ac0d40a0 2024-04-12T19:50:37.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.681+0000 7f22a8a00640 1 -- 172.21.15.32:0/3110599179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a40343f0 con 0x7f22ac0d40a0 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 -- 172.21.15.32:0/3110599179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 msgr2=0x7f22ac0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac0d1950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e040 tx=0x7f22a402bed0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 -- 172.21.15.32:0/3110599179 shutdown_connections 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac0d1950 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 --2- 172.21.15.32:0/3110599179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac15ed30 0x7f22ac0ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 --2- 172.21.15.32:0/3110599179 >> v2:172.21.15.169:3300/0 conn(0x7f22ac0d1570 0x7f22ac0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.682+0000 7f22ab200640 1 -- 172.21.15.32:0/3110599179 >> 172.21.15.32:0/3110599179 conn(0x7f22ac073240 msgr2=0x7f22ac073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.683+0000 7f22ab200640 1 -- 172.21.15.32:0/3110599179 shutdown_connections 2024-04-12T19:50:37.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.683+0000 7f22ab200640 1 -- 172.21.15.32:0/3110599179 wait complete. 2024-04-12T19:50:37.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.684+0000 7f22ab200640 1 Processor -- start 2024-04-12T19:50:37.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.684+0000 7f22ab200640 1 -- start start 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0d1570 0x7f22ac157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 0x7f22ac0b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0d3b80 con 0x7f22ac0d40a0 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0d38a0 con 0x7f22ac15ed30 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.685+0000 7f22ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0d4530 con 0x7f22ac0d1570 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35286/0 (socket says 172.21.15.32:35286) 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 -- 172.21.15.32:0/3293855530 learned_addr learned my addr 172.21.15.32:0/3293855530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:37.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22aa800640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 0x7f22ac0b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9e00640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0d1570 0x7f22ac157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 -- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0d1570 msgr2=0x7f22ac157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0d1570 0x7f22ac157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 -- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 msgr2=0x7f22ac0b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 0x7f22ac0b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22a9400640 1 -- 172.21.15.32:0/3293855530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac0ce210 con 0x7f22ac0d40a0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.686+0000 7f22aa800640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 0x7f22ac0b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22a9400640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f229800d5c0 tx=0x7f229800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980125f0 con 0x7f22ac0d40a0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298019070 con 0x7f22ac0d40a0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac155160 con 0x7f22ac0d40a0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980129d0 con 0x7f22ac0d40a0 2024-04-12T19:50:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.687+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac0cd730 con 0x7f22ac0d40a0 2024-04-12T19:50:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.688+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac0755c0 con 0x7f22ac0d40a0 2024-04-12T19:50:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.689+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2298012b70 con 0x7f22ac0d40a0 2024-04-12T19:50:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.691+0000 7f22a2000640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 0x7f228c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.692+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f229808bf30 con 0x7f22ac0d40a0 2024-04-12T19:50:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.692+0000 7f22a9e00640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 0x7f228c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.692+0000 7f22a9e00640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 0x7f228c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f229c006fd0 tx=0x7f229c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.696+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f229805a360 con 0x7f22ac0d40a0 2024-04-12T19:50:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.900+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22ac15f3b0 con 0x7f22ac0d40a0 2024-04-12T19:50:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.901+0000 7f22a2000640 1 -- 172.21.15.32:0/3293855530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f2298059c60 con 0x7f22ac0d40a0 2024-04-12T19:50:37.901 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 msgr2=0x7f228c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 0x7f228c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f229c006fd0 tx=0x7f229c008040 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 msgr2=0x7f22ac154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f229800d5c0 tx=0x7f229800da80 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 shutdown_connections 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f228c06c8f0 0x7f228c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15ed30 0x7f22ac0b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22ac0d40a0 0x7f22ac154910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 --2- 172.21.15.32:0/3293855530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0d1570 0x7f22ac157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.903+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 >> 172.21.15.32:0/3293855530 conn(0x7f22ac073240 msgr2=0x7f22ac160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.904+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 shutdown_connections 2024-04-12T19:50:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:37.904+0000 7f22ab200640 1 -- 172.21.15.32:0/3293855530 wait complete. 2024-04-12T19:50:37.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:50:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.150+0000 7fc474e00640 1 Processor -- start 2024-04-12T19:50:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.151+0000 7fc474e00640 1 -- start start 2024-04-12T19:50:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.152+0000 7fc474e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc47015a440 0x7fc47015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.152+0000 7fc474e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47015d520 con 0x7fc47006fc80 2024-04-12T19:50:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.152+0000 7fc474e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47015d800 con 0x7fc470072150 2024-04-12T19:50:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.152+0000 7fc474e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc47006e280 con 0x7fc47015a440 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc47015a440 0x7fc47015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc47015a440 0x7fc47015cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35296/0 (socket says 172.21.15.32:35296) 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 -- 172.21.15.32:0/2917104134 learned_addr learned my addr 172.21.15.32:0/2917104134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 -- 172.21.15.32:0/2917104134 >> v1:172.21.15.169:6789/0 conn(0x7fc470072150 legacy=0x7fc47006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 -- 172.21.15.32:0/2917104134 >> v1:172.21.15.32:6789/0 conn(0x7fc47006fc80 legacy=0x7fc4700cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.153+0000 7fc46de00640 1 -- 172.21.15.32:0/2917104134 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47015dae0 con 0x7fc47015a440 2024-04-12T19:50:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.154+0000 7fc46de00640 1 --2- 172.21.15.32:0/2917104134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc47015cef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc47006e400 tx=0x7fc46002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5f52c7e4ee0b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc46d400640 1 -- 172.21.15.32:0/2917104134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460034070 con 0x7fc47015a440 2024-04-12T19:50:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc46d400640 1 -- 172.21.15.32:0/2917104134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc460030840 con 0x7fc47015a440 2024-04-12T19:50:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc46d400640 1 -- 172.21.15.32:0/2917104134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460030c10 con 0x7fc47015a440 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc474e00640 1 -- 172.21.15.32:0/2917104134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 msgr2=0x7fc47015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc474e00640 1 --2- 172.21.15.32:0/2917104134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc47015cef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc47006e400 tx=0x7fc46002b980 comp rx=0 tx=0).stop 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc474e00640 1 -- 172.21.15.32:0/2917104134 shutdown_connections 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc474e00640 1 --2- 172.21.15.32:0/2917104134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc47015cef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.155+0000 7fc474e00640 1 -- 172.21.15.32:0/2917104134 >> 172.21.15.32:0/2917104134 conn(0x7fc4700b2460 msgr2=0x7fc4700b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.156+0000 7fc474e00640 1 -- 172.21.15.32:0/2917104134 shutdown_connections 2024-04-12T19:50:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.156+0000 7fc474e00640 1 -- 172.21.15.32:0/2917104134 wait complete. 2024-04-12T19:50:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.157+0000 7fc474e00640 1 Processor -- start 2024-04-12T19:50:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.158+0000 7fc474e00640 1 -- start start 2024-04-12T19:50:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.158+0000 7fc474e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 0x7fc470143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc474e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 0x7fc4700ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc474e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc474e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc47015ddc0 con 0x7fc47015a440 2024-04-12T19:50:38.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc474e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc47015a820 con 0x7fc470072150 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc474e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc47006f510 con 0x7fc47006fc80 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35300/0 (socket says 172.21.15.32:35300) 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 -- 172.21.15.32:0/1532129833 learned_addr learned my addr 172.21.15.32:0/1532129833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46de00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 0x7fc470143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc467e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 0x7fc4700ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 -- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 msgr2=0x7fc470143050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 0x7fc470143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 -- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 msgr2=0x7fc4700ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 0x7fc4700ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46e800640 1 -- 172.21.15.32:0/1532129833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4701fc400 con 0x7fc47015a440 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc46de00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 0x7fc470143050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.159+0000 7fc467e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 0x7fc4700ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc46e800640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c7b0 tx=0x7fc46800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468004b60 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc468004d00 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4701fc6e0 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800aa90 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.160+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4701fcca0 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.161+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc470068c80 con 0x7fc47015a440 2024-04-12T19:50:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.162+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc46800ac30 con 0x7fc47015a440 2024-04-12T19:50:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.163+0000 7fc466000640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.163+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc46808bed0 con 0x7fc47015a440 2024-04-12T19:50:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.164+0000 7fc46de00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.164+0000 7fc46de00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 0x7fc44c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc460036000 tx=0x7fc46003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.168+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc468020070 con 0x7fc47015a440 2024-04-12T19:50:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.370+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc470144650 con 0x7fc47015a440 2024-04-12T19:50:38.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.371+0000 7fc466000640 1 -- 172.21.15.32:0/1532129833 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc46805a260 con 0x7fc47015a440 2024-04-12T19:50:38.371 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 msgr2=0x7fc44c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 0x7fc44c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc460036000 tx=0x7fc46003b040 comp rx=0 tx=0).stop 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 msgr2=0x7fc4700cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c7b0 tx=0x7fc46800cc70 comp rx=0 tx=0).stop 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 shutdown_connections 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.373+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc47015a440 0x7fc4700cda30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.374+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc470072150 0x7fc4700ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.374+0000 7fc474e00640 1 --2- 172.21.15.32:0/1532129833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc47006fc80 0x7fc470143050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.374+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 >> 172.21.15.32:0/1532129833 conn(0x7fc4700b2460 msgr2=0x7fc47015f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.374+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 shutdown_connections 2024-04-12T19:50:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.374+0000 7fc474e00640 1 -- 172.21.15.32:0/1532129833 wait complete. 2024-04-12T19:50:38.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:50:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.609+0000 7f3389800640 1 Processor -- start 2024-04-12T19:50:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.610+0000 7f3389800640 1 -- start start 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f3389800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 0x7f3384162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f3389800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33840cf110 0x7f33840d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f3389800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33840d0090 con 0x7f33840d1ac0 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f3389800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f33840d1f50 con 0x7f33840cf110 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f3389800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3384163440 con 0x7f3384162020 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f337be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33840cf110 0x7f33840d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f337be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33840cf110 0x7f33840d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.611+0000 7f337be00640 1 -- 172.21.15.32:0/823065371 learned_addr learned my addr 172.21.15.32:0/823065371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f3383e00640 1 --2- 172.21.15.32:0/823065371 >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 0x7f3384162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f337be00640 1 -- 172.21.15.32:0/823065371 >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 msgr2=0x7f3384162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f337be00640 1 --2- 172.21.15.32:0/823065371 >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 0x7f3384162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f337be00640 1 -- 172.21.15.32:0/823065371 >> v1:172.21.15.169:6789/0 conn(0x7f33840d1ac0 legacy=0x7f33840b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f337be00640 1 -- 172.21.15.32:0/823065371 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33840cfad0 con 0x7f33840cf110 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.612+0000 7f3383e00640 1 --2- 172.21.15.32:0/823065371 >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 0x7f3384162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.613+0000 7f337be00640 1 --2- 172.21.15.32:0/823065371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33840cf110 0x7f33840d1580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f338408cd90 tx=0x7f336c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee8a48f3b77d0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.613+0000 7f3383400640 1 -- 172.21.15.32:0/823065371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c035020 con 0x7f33840cf110 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.613+0000 7f3383400640 1 -- 172.21.15.32:0/823065371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f336c031870 con 0x7f33840cf110 2024-04-12T19:50:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.613+0000 7f3383400640 1 -- 172.21.15.32:0/823065371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c031c40 con 0x7f33840cf110 2024-04-12T19:50:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 -- 172.21.15.32:0/823065371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33840cf110 msgr2=0x7f33840d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 --2- 172.21.15.32:0/823065371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33840cf110 0x7f33840d1580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f338408cd90 tx=0x7f336c02b910 comp rx=0 tx=0).stop 2024-04-12T19:50:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 -- 172.21.15.32:0/823065371 shutdown_connections 2024-04-12T19:50:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 --2- 172.21.15.32:0/823065371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33840cf110 0x7f33840d1580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 --2- 172.21.15.32:0/823065371 >> v2:172.21.15.169:3300/0 conn(0x7f3384162020 0x7f3384162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 -- 172.21.15.32:0/823065371 >> 172.21.15.32:0/823065371 conn(0x7f338406cf00 msgr2=0x7f338406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 -- 172.21.15.32:0/823065371 shutdown_connections 2024-04-12T19:50:38.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.614+0000 7f3389800640 1 -- 172.21.15.32:0/823065371 wait complete. 2024-04-12T19:50:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.616+0000 7f3389800640 1 Processor -- start 2024-04-12T19:50:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.616+0000 7f3389800640 1 -- start start 2024-04-12T19:50:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33840cf110 0x7f338414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33840d1ac0 0x7f3384160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33840cf880 con 0x7f3384162020 2024-04-12T19:50:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33840b7b30 con 0x7f33840cf110 2024-04-12T19:50:38.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.617+0000 7f3389800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33840d2b90 con 0x7f33840d1ac0 2024-04-12T19:50:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3383e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33840cf110 0x7f338414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35326/0 (socket says 172.21.15.32:35326) 2024-04-12T19:50:38.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 -- 172.21.15.32:0/710428207 learned_addr learned my addr 172.21.15.32:0/710428207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f337be00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33840d1ac0 0x7f3384160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 -- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33840d1ac0 msgr2=0x7f3384160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33840d1ac0 0x7f3384160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 -- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33840cf110 msgr2=0x7f338414cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33840cf110 0x7f338414cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.618+0000 7f3388e00640 1 -- 172.21.15.32:0/710428207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338415b550 con 0x7f3384162020 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3388e00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f337c00c7b0 tx=0x7f337c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c017070 con 0x7f3384162020 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337c003da0 con 0x7f3384162020 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c0052b0 con 0x7f3384162020 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338415b830 con 0x7f3384162020 2024-04-12T19:50:38.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.619+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338415b200 con 0x7f3384162020 2024-04-12T19:50:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.620+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338415ba00 con 0x7f3384162020 2024-04-12T19:50:38.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.621+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f337c005450 con 0x7f3384162020 2024-04-12T19:50:38.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.622+0000 7f3380c00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 0x7f336006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.622+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f337c08aee0 con 0x7f3384162020 2024-04-12T19:50:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.623+0000 7f3383e00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 0x7f336006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.623+0000 7f3383e00640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 0x7f336006ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33840b74d0 tx=0x7f3374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:38.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.627+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f337c058d50 con 0x7f3384162020 2024-04-12T19:50:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.873+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33840b7650 con 0x7f3384162020 2024-04-12T19:50:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.874+0000 7f3380c00640 1 -- 172.21.15.32:0/710428207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f337c05d050 con 0x7f3384162020 2024-04-12T19:50:38.875 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:38.875 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:50:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 msgr2=0x7f336006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 0x7f336006ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33840b74d0 tx=0x7f3374008040 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 msgr2=0x7f3384161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f337c00c7b0 tx=0x7f337c00cc70 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 shutdown_connections 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f336006c920 0x7f336006ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3384162020 0x7f3384161230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33840d1ac0 0x7f3384160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 --2- 172.21.15.32:0/710428207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33840cf110 0x7f338414cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 >> 172.21.15.32:0/710428207 conn(0x7f338406cf00 msgr2=0x7f33840d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:38.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 shutdown_connections 2024-04-12T19:50:38.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:38.877+0000 7f3389800640 1 -- 172.21.15.32:0/710428207 wait complete. 2024-04-12T19:50:38.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.114+0000 7fa57d600640 1 Processor -- start 2024-04-12T19:50:39.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.114+0000 7fa57d600640 1 -- start start 2024-04-12T19:50:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.116+0000 7fa57d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa57815b480 0x7fa5780cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.116+0000 7fa57d600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa57815b860 0x7fa5780b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.116+0000 7fa57d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57815f2b0 con 0x7fa5780cdde0 2024-04-12T19:50:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.116+0000 7fa57d600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa57815f060 con 0x7fa57815b860 2024-04-12T19:50:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.116+0000 7fa57d600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa57815fec0 con 0x7fa57815b480 2024-04-12T19:50:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa57815b860 0x7fa5780b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa57815b860 0x7fa5780b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35336/0 (socket says 172.21.15.32:35336) 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 -- 172.21.15.32:0/213285774 learned_addr learned my addr 172.21.15.32:0/213285774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa577e00640 1 --2- 172.21.15.32:0/213285774 >> v2:172.21.15.169:3300/0 conn(0x7fa57815b480 0x7fa5780cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 -- 172.21.15.32:0/213285774 >> v2:172.21.15.169:3300/0 conn(0x7fa57815b480 msgr2=0x7fa5780cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 --2- 172.21.15.32:0/213285774 >> v2:172.21.15.169:3300/0 conn(0x7fa57815b480 0x7fa5780cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 -- 172.21.15.32:0/213285774 >> v1:172.21.15.169:6789/0 conn(0x7fa5780cdde0 legacy=0x7fa57806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.117+0000 7fa57cc00640 1 -- 172.21.15.32:0/213285774 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57815f590 con 0x7fa57815b860 2024-04-12T19:50:39.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.119+0000 7fa57cc00640 1 --2- 172.21.15.32:0/213285774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5780b3fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa57806c720 tx=0x7fa57002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4809370009d0367d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.119+0000 7fa577400640 1 -- 172.21.15.32:0/213285774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57003b070 con 0x7fa57815b860 2024-04-12T19:50:39.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.119+0000 7fa577400640 1 -- 172.21.15.32:0/213285774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa570036440 con 0x7fa57815b860 2024-04-12T19:50:39.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.119+0000 7fa577400640 1 -- 172.21.15.32:0/213285774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570036810 con 0x7fa57815b860 2024-04-12T19:50:39.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 -- 172.21.15.32:0/213285774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 msgr2=0x7fa5780b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 --2- 172.21.15.32:0/213285774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5780b3fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa57806c720 tx=0x7fa57002e170 comp rx=0 tx=0).stop 2024-04-12T19:50:39.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 -- 172.21.15.32:0/213285774 shutdown_connections 2024-04-12T19:50:39.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 --2- 172.21.15.32:0/213285774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5780b3fc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 --2- 172.21.15.32:0/213285774 >> v2:172.21.15.169:3300/0 conn(0x7fa57815b480 0x7fa5780cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 -- 172.21.15.32:0/213285774 >> 172.21.15.32:0/213285774 conn(0x7fa57806b930 msgr2=0x7fa57806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 -- 172.21.15.32:0/213285774 shutdown_connections 2024-04-12T19:50:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.120+0000 7fa57d600640 1 -- 172.21.15.32:0/213285774 wait complete. 2024-04-12T19:50:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.122+0000 7fa57d600640 1 Processor -- start 2024-04-12T19:50:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.122+0000 7fa57d600640 1 -- start start 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5780cdde0 0x7fa578153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 0x7fa5780c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5780cd550 con 0x7fa57815b860 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa57815fb50 con 0x7fa57815b480 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa57806f4f0 con 0x7fa5780cdde0 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35350/0 (socket says 172.21.15.32:35350) 2024-04-12T19:50:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.123+0000 7fa57cc00640 1 -- 172.21.15.32:0/868469309 learned_addr learned my addr 172.21.15.32:0/868469309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa577e00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5780cdde0 0x7fa578153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa56f400640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 0x7fa5780c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 -- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5780cdde0 msgr2=0x7fa578153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5780cdde0 0x7fa578153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 -- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 msgr2=0x7fa5780c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 0x7fa5780c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 -- 172.21.15.32:0/868469309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa578154540 con 0x7fa57815b860 2024-04-12T19:50:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa56f400640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 0x7fa5780c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.124+0000 7fa57cc00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa57002e1b0 tx=0x7fa5700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.125+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570045020 con 0x7fa57815b860 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.125+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5700346e0 con 0x7fa57815b860 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.125+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa578154820 con 0x7fa57815b860 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.125+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57003b040 con 0x7fa57815b860 2024-04-12T19:50:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.125+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa578154de0 con 0x7fa57815b860 2024-04-12T19:50:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.126+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5781552c0 con 0x7fa57815b860 2024-04-12T19:50:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.127+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa570048020 con 0x7fa57815b860 2024-04-12T19:50:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.128+0000 7fa574c00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 0x7fa55406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.128+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa5700b03c0 con 0x7fa57815b860 2024-04-12T19:50:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.128+0000 7fa577e00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 0x7fa55406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.129+0000 7fa577e00640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 0x7fa55406ee30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa578068b60 tx=0x7fa56800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.133+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5781552c0 con 0x7fa57815b860 2024-04-12T19:50:39.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.336+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa578158730 con 0x7fa57815b860 2024-04-12T19:50:39.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.339+0000 7fa574c00640 1 -- 172.21.15.32:0/868469309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7fa57004f030 con 0x7fa57815b860 2024-04-12T19:50:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:39.340 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 msgr2=0x7fa55406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 0x7fa55406ee30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa578068b60 tx=0x7fa56800b040 comp rx=0 tx=0).stop 2024-04-12T19:50:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 msgr2=0x7fa5781518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa57002e1b0 tx=0x7fa5700046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 shutdown_connections 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa55406c970 0x7fa55406ee30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57815b860 0x7fa5781518c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa57815b480 0x7fa5780c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 --2- 172.21.15.32:0/868469309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5780cdde0 0x7fa578153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 >> 172.21.15.32:0/868469309 conn(0x7fa57806b930 msgr2=0x7fa5780b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 shutdown_connections 2024-04-12T19:50:39.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.342+0000 7fa57d600640 1 -- 172.21.15.32:0/868469309 wait complete. 2024-04-12T19:50:39.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:50:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.574+0000 7f0850c00640 1 Processor -- start 2024-04-12T19:50:39.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.574+0000 7f0850c00640 1 -- start start 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.576+0000 7f0850c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f084c162020 0x7f084c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.576+0000 7f0850c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.576+0000 7f0850c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f084c0d0090 con 0x7f084c0d1ac0 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.576+0000 7f0850c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f084c0d1f50 con 0x7f084c162020 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.576+0000 7f0850c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f084c163440 con 0x7f084c0cf110 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f084c0d1ac0 0x7f084c0b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51962/0 (socket says 172.21.15.32:51962) 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084be00640 1 -- 172.21.15.32:0/3423213884 learned_addr learned my addr 172.21.15.32:0/3423213884 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084b400640 1 --2- 172.21.15.32:0/3423213884 >> v2:172.21.15.169:3300/0 conn(0x7f084c162020 0x7f084c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f0843e00640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654099629 0 0) 0x7f0844001490 con 0x7f084c0d1ac0 2024-04-12T19:50:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0830003520 con 0x7f084c0d1ac0 2024-04-12T19:50:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084b400640 1 -- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 msgr2=0x7f084c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084b400640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084b400640 1 -- 172.21.15.32:0/3423213884 >> v1:172.21.15.32:6789/0 conn(0x7f084c0d1ac0 legacy=0x7f084c0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3226319642 0 0) 0x7f0830003520 con 0x7f084c0d1ac0 2024-04-12T19:50:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.577+0000 7f084b400640 1 -- 172.21.15.32:0/3423213884 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c0cfad0 con 0x7f084c162020 2024-04-12T19:50:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.579+0000 7f084b400640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c162020 0x7f084c162f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f083c0097a0 tx=0x7f083c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d16ab784256795 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.579+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c037070 con 0x7f084c162020 2024-04-12T19:50:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.579+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c030510 con 0x7f084c162020 2024-04-12T19:50:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.579+0000 7f084aa00640 1 -- 172.21.15.32:0/3423213884 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844001bc0 con 0x7f084c162020 2024-04-12T19:50:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 -- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c162020 msgr2=0x7f084c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c162020 0x7f084c162f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f083c0097a0 tx=0x7f083c02b790 comp rx=0 tx=0).stop 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 -- 172.21.15.32:0/3423213884 shutdown_connections 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 --2- 172.21.15.32:0/3423213884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c162020 0x7f084c162f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 -- 172.21.15.32:0/3423213884 >> 172.21.15.32:0/3423213884 conn(0x7f084c06cf00 msgr2=0x7f084c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 -- 172.21.15.32:0/3423213884 shutdown_connections 2024-04-12T19:50:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.580+0000 7f0850c00640 1 -- 172.21.15.32:0/3423213884 wait complete. 2024-04-12T19:50:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.582+0000 7f0850c00640 1 Processor -- start 2024-04-12T19:50:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.582+0000 7f0850c00640 1 -- start start 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 0x7f084c1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f084c0cf880 con 0x7f084c162020 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f084c0b7b30 con 0x7f084c0d1ac0 2024-04-12T19:50:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.583+0000 7f0850c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f084c0d2b90 con 0x7f084c0cf110 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35356/0 (socket says 172.21.15.32:35356) 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 -- 172.21.15.32:0/1588944220 learned_addr learned my addr 172.21.15.32:0/1588944220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084b400640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f0843e00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 0x7f084c1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 -- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 msgr2=0x7f084c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 -- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 msgr2=0x7f084c1f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 0x7f084c1f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 -- 172.21.15.32:0/1588944220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c15e2e0 con 0x7f084c162020 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f0843e00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 0x7f084c1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.584+0000 7f084be00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f084400cc30 tx=0x7f084400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.585+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844017070 con 0x7f084c162020 2024-04-12T19:50:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.585+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084400ce40 con 0x7f084c162020 2024-04-12T19:50:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.585+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c15e5c0 con 0x7f084c162020 2024-04-12T19:50:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.585+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844012640 con 0x7f084c162020 2024-04-12T19:50:39.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.585+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c15eb80 con 0x7f084c162020 2024-04-12T19:50:39.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.586+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c0755c0 con 0x7f084c162020 2024-04-12T19:50:39.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.587+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084401a070 con 0x7f084c162020 2024-04-12T19:50:39.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.588+0000 7f0843400640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 0x7f082806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:39.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.588+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f084408bb50 con 0x7f084c162020 2024-04-12T19:50:39.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.588+0000 7f084b400640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 0x7f082806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:39.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.589+0000 7f084b400640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 0x7f082806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f083c034000 tx=0x7f083c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:39.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.592+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f084c0d0090 con 0x7f084c162020 2024-04-12T19:50:39.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:50:39.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:50:39.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.843+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f084c163060 con 0x7f084c162020 2024-04-12T19:50:39.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.844+0000 7f0843400640 1 -- 172.21.15.32:0/1588944220 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f0844059ee0 con 0x7f084c162020 2024-04-12T19:50:39.845 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:39.845 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:50:39.845 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:50:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 msgr2=0x7f082806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 0x7f082806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f083c034000 tx=0x7f083c02f000 comp rx=0 tx=0).stop 2024-04-12T19:50:39.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 msgr2=0x7f084c0ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f084400cc30 tx=0x7f084400a430 comp rx=0 tx=0).stop 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 shutdown_connections 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082806c920 0x7f082806ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f084c162020 0x7f084c0ca2c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f084c0d1ac0 0x7f084c1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 --2- 172.21.15.32:0/1588944220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f084c0cf110 0x7f084c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 >> 172.21.15.32:0/1588944220 conn(0x7f084c06cf00 msgr2=0x7f084c0b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 shutdown_connections 2024-04-12T19:50:39.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:39.847+0000 7f0850c00640 1 -- 172.21.15.32:0/1588944220 wait complete. 2024-04-12T19:50:40.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-12T19:50:24.149610+0000', 'modified': '2024-04-12T19:50:27.198866+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/1878337294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 1878337294}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 1878337294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4291, 'qdb_cluster': [4291]} 2024-04-12T19:50:40.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-12T19:50:40.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-12T19:50:40.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.503+0000 7f5655200640 1 Processor -- start 2024-04-12T19:50:40.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.504+0000 7f5655200640 1 -- start start 2024-04-12T19:50:40.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f5650162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5650162b70 0x7f56500d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56500d1a90 0x7f56500b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f56500d2e50 con 0x7f56500d1a90 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f56500b78f0 con 0x7f5650162b70 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.505+0000 7f5655200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5650163010 con 0x7f565015eab0 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56500d1a90 0x7f56500b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56500d1a90 0x7f56500b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35372/0 (socket says 172.21.15.32:35372) 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 -- 172.21.15.32:0/3170050871 learned_addr learned my addr 172.21.15.32:0/3170050871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564ea00640 1 --2- 172.21.15.32:0/3170050871 >> v2:172.21.15.169:3300/0 conn(0x7f5650162b70 0x7f56500d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564f400640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f5650162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 -- 172.21.15.32:0/3170050871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 msgr2=0x7f5650162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f5650162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 -- 172.21.15.32:0/3170050871 >> v2:172.21.15.169:3300/0 conn(0x7f5650162b70 msgr2=0x7f56500d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 --2- 172.21.15.32:0/3170050871 >> v2:172.21.15.169:3300/0 conn(0x7f5650162b70 0x7f56500d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.506+0000 7f564fe00640 1 -- 172.21.15.32:0/3170050871 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56501632f0 con 0x7f56500d1a90 2024-04-12T19:50:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.508+0000 7f564fe00640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f56500b6240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56500d58c0 tx=0x7f564802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c07cbe877adbca server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.508+0000 7f564e000640 1 -- 172.21.15.32:0/3170050871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564802ca70 con 0x7f56500d1a90 2024-04-12T19:50:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.508+0000 7f564e000640 1 -- 172.21.15.32:0/3170050871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f564802cc10 con 0x7f56500d1a90 2024-04-12T19:50:40.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.508+0000 7f564e000640 1 -- 172.21.15.32:0/3170050871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56480343f0 con 0x7f56500d1a90 2024-04-12T19:50:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 -- 172.21.15.32:0/3170050871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 msgr2=0x7f56500b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f56500b6240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56500d58c0 tx=0x7f564802bed0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 -- 172.21.15.32:0/3170050871 shutdown_connections 2024-04-12T19:50:40.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f56500b6240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 --2- 172.21.15.32:0/3170050871 >> v2:172.21.15.169:3300/0 conn(0x7f5650162b70 0x7f56500d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 --2- 172.21.15.32:0/3170050871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f5650162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 -- 172.21.15.32:0/3170050871 >> 172.21.15.32:0/3170050871 conn(0x7f5650073240 msgr2=0x7f5650073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 -- 172.21.15.32:0/3170050871 shutdown_connections 2024-04-12T19:50:40.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.509+0000 7f5655200640 1 -- 172.21.15.32:0/3170050871 wait complete. 2024-04-12T19:50:40.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.511+0000 7f5655200640 1 Processor -- start 2024-04-12T19:50:40.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.511+0000 7f5655200640 1 -- start start 2024-04-12T19:50:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f565015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 0x7f56501527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56500ba590 con 0x7f56500d1a90 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f565015ee90 con 0x7f5650162b70 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f5655200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56500ba710 con 0x7f565015eab0 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f564f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f564f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35386/0 (socket says 172.21.15.32:35386) 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.512+0000 7f564f400640 1 -- 172.21.15.32:0/1576638311 learned_addr learned my addr 172.21.15.32:0/1576638311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564ea00640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f565015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564fe00640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 0x7f56501527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 -- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 msgr2=0x7f565015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f565015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 -- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 msgr2=0x7f56501527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 0x7f56501527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 -- 172.21.15.32:0/1576638311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565015b180 con 0x7f56500d1a90 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564fe00640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 0x7f56501527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f564f400640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f564000d800 tx=0x7f564000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f5647400640 1 -- 172.21.15.32:0/1576638311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640018020 con 0x7f56500d1a90 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f5647400640 1 -- 172.21.15.32:0/1576638311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56400096a0 con 0x7f56500d1a90 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f5647400640 1 -- 172.21.15.32:0/1576638311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640009a70 con 0x7f56500d1a90 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.513+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565015b420 con 0x7f56500d1a90 2024-04-12T19:50:40.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.514+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565015b9e0 con 0x7f56500d1a90 2024-04-12T19:50:40.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.516+0000 7f5647400640 1 -- 172.21.15.32:0/1576638311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f564001b020 con 0x7f56500d1a90 2024-04-12T19:50:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.517+0000 7f5647400640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.517+0000 7f5647400640 1 -- 172.21.15.32:0/1576638311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f564008bd40 con 0x7f56500d1a90 2024-04-12T19:50:40.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.517+0000 7f564ea00640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.518+0000 7f564ea00640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 0x7f563006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f563c006fd0 tx=0x7f563c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.518+0000 7f56595fe740 1 Processor -- start 2024-04-12T19:50:40.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.518+0000 7f56595fe740 1 -- start start 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1ad9d0 0x5580dc1ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5580dc112990 con 0x5580dc1d4b90 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580dc0b9530 con 0x5580dc1d47b0 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5580dc10e950 con 0x5580dc1ad9d0 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35396/0 (socket says 172.21.15.32:35396) 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 -- 172.21.15.32:0/1769388204 learned_addr learned my addr 172.21.15.32:0/1769388204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564f400640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564fe00640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1ad9d0 0x5580dc1ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 -- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1ad9d0 msgr2=0x5580dc1ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1ad9d0 0x5580dc1ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 -- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 msgr2=0x5580dc1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564ea00640 1 -- 172.21.15.32:0/1769388204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580dbe252d0 con 0x5580dc1d4b90 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.519+0000 7f564f400640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.520+0000 7f564ea00640 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b590 tx=0x7f563c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c938422ff45f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.520+0000 7f563be00640 1 -- 172.21.15.32:0/1769388204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c01b050 con 0x5580dc1d4b90 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.520+0000 7f563be00640 1 -- 172.21.15.32:0/1769388204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563c025070 con 0x5580dc1d4b90 2024-04-12T19:50:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.520+0000 7f563be00640 1 -- 172.21.15.32:0/1769388204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c014d80 con 0x5580dc1d4b90 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 -- 172.21.15.32:0/1769388204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 msgr2=0x5580dc1ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b590 tx=0x7f563c012000 comp rx=0 tx=0).stop 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 -- 172.21.15.32:0/1769388204 shutdown_connections 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1ad9d0 0x5580dc1ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1ad490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 --2- 172.21.15.32:0/1769388204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 -- 172.21.15.32:0/1769388204 >> 172.21.15.32:0/1769388204 conn(0x5580dc1a9a70 msgr2=0x5580dc1f25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 -- 172.21.15.32:0/1769388204 shutdown_connections 2024-04-12T19:50:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.521+0000 7f56595fe740 1 -- 172.21.15.32:0/1769388204 wait complete. 2024-04-12T19:50:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.522+0000 7f56595fe740 1 -- start start 2024-04-12T19:50:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.522+0000 7f56595fe740 1 Processor -- start 2024-04-12T19:50:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 0x5580dc1f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5580dc1a78a0 con 0x5580dc1d4b90 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580dc0cc8f0 con 0x5580dc1d47b0 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5580dc1ade50 con 0x5580dc1f0620 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f56595fe740 10 client.0 _ll_register_callbacks cb 0x5580dc0cc2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f564ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f564ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35402/0 (socket says 172.21.15.32:35402) 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f564ea00640 1 -- 172.21.15.32:0/2991523476 learned_addr learned my addr 172.21.15.32:0/2991523476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f564f400640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.523+0000 7f564fe00640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 0x5580dc1f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 -- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 msgr2=0x5580dc1f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 0x5580dc1f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 -- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 msgr2=0x5580dc1eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 -- 172.21.15.32:0/2991523476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580dbeef6b0 con 0x5580dc1d4b90 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564fe00640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 0x5580dc1f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f564ea00640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f563c021000 tx=0x7f563c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab00a87dc46ba95f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f5638c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c006240 con 0x5580dc1d4b90 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f563c0063e0 con 0x5580dc1d4b90 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.524+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563c025070 con 0x5580dc1d4b90 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f563c0065e0 con 0x5580dc1d4b90 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 10 client.4599 fetch_fsmap learned FSMap version 8 2024-04-12T19:50:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5580dc1f10e0 con 0x5580dc1d4b90 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563c025070 con 0x5580dc1d4b90 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f563c040020 con 0x5580dc1d4b90 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 10 client.4599 fetch_fsmap finished waiting for FSMap version 8 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 10 client.4599 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 20 client.4599 populate_metadata read hostname 'smithi032' 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1ce830 0x5580dc202810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 4 client.4599 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.525+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 --> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580dc0b8b40 con 0x5580dc1ce830 2024-04-12T19:50:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.526+0000 7f564f400640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1ce830 0x5580dc202810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.527+0000 7f564f400640 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1ce830 0x5580dc202810 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314433c6a43640b6 server_cookie=9e8903c3f08ad0d1 in_seq=0 out_seq=0 2024-04-12T19:50:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.527+0000 7f5638c00640 10 client.4599 ms_handle_connect on v2:172.21.15.169:6832/1878337294 2024-04-12T19:50:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.529+0000 7f5638c00640 1 -- 172.21.15.32:0/2991523476 <== mds.0 v2:172.21.15.169:6832/1878337294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580dc0b8b40 con 0x5580dc1ce830 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.529+0000 7f5638c00640 10 client.4599 handle_command_reply: tid=0 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.529+0000 7f56595fe740 1 client.4599 shutdown 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 msgr2=0x5580dc1f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f563c021000 tx=0x7f563c004730 comp rx=0 tx=0).stop 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 shutdown_connections 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1ce830 0x5580dc202810 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1f0620 0x5580dc1f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d4b90 0x5580dc1f64c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 --2- 172.21.15.32:0/2991523476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d47b0 0x5580dc1eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f5638c00640 0 client.4599 ms_handle_reset on v2:172.21.15.169:6832/1878337294 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 >> 172.21.15.32:0/2991523476 conn(0x5580dc1ad9d0 msgr2=0x5580dc1a66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 shutdown_connections 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 1 -- 172.21.15.32:0/2991523476 wait complete. 2024-04-12T19:50:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.530+0000 7f56595fe740 20 client.4599 trim_cache size 0 max 16384 2024-04-12T19:50:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.550+0000 7f56595fe740 1 Processor -- start 2024-04-12T19:50:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.550+0000 7f56595fe740 1 -- start start 2024-04-12T19:50:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d4b90 0x5580dc1f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 0x5580dc1a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56501611e0 con 0x5580dc1d47b0 2024-04-12T19:50:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56501579f0 con 0x5580dc1f0620 2024-04-12T19:50:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.551+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5650160b90 con 0x5580dc1d4b90 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35408/0 (socket says 172.21.15.32:35408) 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 -- 172.21.15.32:0/818046387 learned_addr learned my addr 172.21.15.32:0/818046387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564fe00640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 0x5580dc1a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564ea00640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d4b90 0x5580dc1f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 -- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d4b90 msgr2=0x5580dc1f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d4b90 0x5580dc1f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 -- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 msgr2=0x5580dc1a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 0x5580dc1a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564f400640 1 -- 172.21.15.32:0/818046387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580dc1f6810 con 0x5580dc1d47b0 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.552+0000 7f564fe00640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 0x5580dc1a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f564f400640 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5640059720 tx=0x7f564000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d6e5961d9ac872 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f563b400640 1 -- 172.21.15.32:0/818046387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564000b3c0 con 0x5580dc1d47b0 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f563b400640 1 -- 172.21.15.32:0/818046387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5640057b10 con 0x5580dc1d47b0 2024-04-12T19:50:40.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f563b400640 1 -- 172.21.15.32:0/818046387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640024de0 con 0x5580dc1d47b0 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f56595fe740 1 -- 172.21.15.32:0/818046387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 msgr2=0x5580dc1f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.553+0000 7f56595fe740 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5640059720 tx=0x7f564000d750 comp rx=0 tx=0).stop 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 -- 172.21.15.32:0/818046387 shutdown_connections 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1f0620 0x5580dc1a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d4b90 0x5580dc1f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 --2- 172.21.15.32:0/818046387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1d47b0 0x5580dc1f62d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 -- 172.21.15.32:0/818046387 >> 172.21.15.32:0/818046387 conn(0x5580dc1ad9d0 msgr2=0x5580dc1cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 -- 172.21.15.32:0/818046387 shutdown_connections 2024-04-12T19:50:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 -- 172.21.15.32:0/818046387 wait complete. 2024-04-12T19:50:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.554+0000 7f56595fe740 1 -- start start 2024-04-12T19:50:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.555+0000 7f56595fe740 1 Processor -- start 2024-04-12T19:50:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d47b0 0x5580dc2554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d4b90 0x5580dc1a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5580dc1ee6d0 con 0x5580dc1f0620 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580dc1eea40 con 0x5580dc1d4b90 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5580dc206360 con 0x5580dc1d47b0 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f56595fe740 10 client.0 _ll_register_callbacks cb 0x7f563c005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35410/0 (socket says 172.21.15.32:35410) 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 -- 172.21.15.32:0/2140501445 learned_addr learned my addr 172.21.15.32:0/2140501445 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564ea00640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d4b90 0x5580dc1a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564f400640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d47b0 0x5580dc2554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 -- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d47b0 msgr2=0x5580dc2554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d47b0 0x5580dc2554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 -- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d4b90 msgr2=0x5580dc1a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d4b90 0x5580dc1a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.556+0000 7f564fe00640 1 -- 172.21.15.32:0/2140501445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580dc2056a0 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.557+0000 7f564fe00640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5648036040 tx=0x7f5648004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f48d58b107de8d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.557+0000 7f5638c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.557+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648009510 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.557+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f564803eac0 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.557+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5640057b10 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f5648050050 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 10 client.4605 fetch_fsmap learned FSMap version 8 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5580dc1a6ef0 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5640057b10 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f5648053020 con 0x5580dc1f0620 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 10 client.4605 fetch_fsmap finished waiting for FSMap version 8 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 10 client.4605 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 20 client.4605 populate_metadata read hostname 'smithi032' 2024-04-12T19:50:40.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1d1370 0x5580dc1d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 4 client.4605 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-12T19:50:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.558+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 --> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] -- command(tid 0: {"prefix": "status"}) -- 0x5580dc1203b0 con 0x5580dc1d1370 2024-04-12T19:50:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.559+0000 7f564f400640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1d1370 0x5580dc1d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.559+0000 7f564f400640 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1d1370 0x5580dc1d1750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c02f55c9c81385 server_cookie=7eed658a2b9703fa in_seq=0 out_seq=0 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.559+0000 7f5638c00640 10 client.4605 ms_handle_connect on v2:172.21.15.169:6832/1878337294 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f5638c00640 1 -- 172.21.15.32:0/2140501445 <== mds.0 v2:172.21.15.169:6832/1878337294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5580dc1203b0 con 0x5580dc1d1370 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f5638c00640 10 client.4605 handle_command_reply: tid=0 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 client.4605 shutdown 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 msgr2=0x5580dc1a7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5648036040 tx=0x7f5648004ab0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 shutdown_connections 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:6832/1878337294,v1:172.21.15.169:6834/1878337294] conn(0x5580dc1d1370 0x5580dc1d1750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5580dc1f0620 0x5580dc1a7940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580dc1d4b90 0x5580dc1a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f5638c00640 0 client.4605 ms_handle_reset on v2:172.21.15.169:6832/1878337294 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 --2- 172.21.15.32:0/2140501445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5580dc1d47b0 0x5580dc2554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 >> 172.21.15.32:0/2140501445 conn(0x5580dc1ad9d0 msgr2=0x5580dc1cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 shutdown_connections 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.560+0000 7f56595fe740 1 -- 172.21.15.32:0/2140501445 wait complete. 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f56595fe740 20 client.4605 trim_cache size 0 max 16384 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:50:40.563 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "47426077-081d-47b8-bfde-ce036e9b4cf8", 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4291, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 15.891749592, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 22, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 22, 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 21.980865832999999 2024-04-12T19:50:40.564 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 msgr2=0x7f563006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 0x7f563006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f563c006fd0 tx=0x7f563c008040 comp rx=0 tx=0).stop 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 msgr2=0x7f5650158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f564000d800 tx=0x7f564000dcc0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 shutdown_connections 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f563006c8f0 0x7f563006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5650162b70 0x7f56501527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f565015eab0 0x7f565015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 --2- 172.21.15.32:0/1576638311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56500d1a90 0x7f5650158f50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:40.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.561+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 >> 172.21.15.32:0/1576638311 conn(0x7f5650073240 msgr2=0x7f56500c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:40.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.562+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 shutdown_connections 2024-04-12T19:50:40.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:40.562+0000 7f5655200640 1 -- 172.21.15.32:0/1576638311 wait complete. 2024-04-12T19:50:40.582 INFO:teuthology.run_tasks:Running task sequential... 2024-04-12T19:50:40.594 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-12T19:50:40.594 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-12T19:50:40.594 INFO:teuthology.task.print:Enabling mgr modules 2024-04-12T19:50:40.594 INFO:teuthology.run_tasks:Running task kclient... 2024-04-12T19:50:40.622 INFO:tasks.kclient:Mounting kernel clients... 2024-04-12T19:50:40.622 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-12T19:50:40.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:40.623 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:50:40.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:40.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-12T19:50:40.782 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-12T19:50:40.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:50:40.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:40.786 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:50:40.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:40.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-12T19:50:40.939 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-12T19:50:40.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:50:40.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:40.943 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:50:41.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:41.005 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-12T19:50:41.106 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-12T19:50:41.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:50:41.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:41.110 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:50:41.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:41.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-12T19:50:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-12T19:50:41.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:50:41.261 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-12T19:50:41.261 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T19:50:41.261 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T19:50:41.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-12T19:50:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.488+0000 7f5e12400640 1 Processor -- start 2024-04-12T19:50:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.488+0000 7f5e12400640 1 -- start start 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e12400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c162020 0x7f5e0c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e12400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e0c0cf110 0x7f5e0c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e12400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0c0d0090 con 0x7f5e0c0d1ac0 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e12400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5e0c0d1f50 con 0x7f5e0c0cf110 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e12400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c163440 con 0x7f5e0c162020 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e11a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5e0c0d1ac0 0x7f5e0c0b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51978/0 (socket says 172.21.15.32:51978) 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e0be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e0c0cf110 0x7f5e0c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e11a00640 1 -- 172.21.15.32:0/3816761126 learned_addr learned my addr 172.21.15.32:0/3816761126 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.490+0000 7f5e11000640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c162020 0x7f5e0c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0b400640 1 -- 172.21.15.32:0/3816761126 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566061775 0 0) 0x7f5e04001110 con 0x7f5e0c0d1ac0 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0b400640 1 -- 172.21.15.32:0/3816761126 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df0003520 con 0x7f5e0c0d1ac0 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0be00640 1 -- 172.21.15.32:0/3816761126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c162020 msgr2=0x7f5e0c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0be00640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c162020 0x7f5e0c162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0be00640 1 -- 172.21.15.32:0/3816761126 >> v1:172.21.15.32:6789/0 conn(0x7f5e0c0d1ac0 legacy=0x7f5e0c0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.491+0000 7f5e0be00640 1 -- 172.21.15.32:0/3816761126 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c0cfad0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.492+0000 7f5e0be00640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c0d1580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5df40097a0 tx=0x7f5df402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51da33de51fb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.492+0000 7f5e0b400640 1 -- 172.21.15.32:0/3816761126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4035020 con 0x7f5e0c0cf110 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.492+0000 7f5e0b400640 1 -- 172.21.15.32:0/3816761126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df4031830 con 0x7f5e0c0cf110 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.492+0000 7f5e0b400640 1 -- 172.21.15.32:0/3816761126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4031c00 con 0x7f5e0c0cf110 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 -- 172.21.15.32:0/3816761126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 msgr2=0x7f5e0c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c0d1580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5df40097a0 tx=0x7f5df402b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 -- 172.21.15.32:0/3816761126 shutdown_connections 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c0d1580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 --2- 172.21.15.32:0/3816761126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c162020 0x7f5e0c162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 -- 172.21.15.32:0/3816761126 >> 172.21.15.32:0/3816761126 conn(0x7f5e0c06cf00 msgr2=0x7f5e0c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 -- 172.21.15.32:0/3816761126 shutdown_connections 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.493+0000 7f5e12400640 1 -- 172.21.15.32:0/3816761126 wait complete. 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.495+0000 7f5e12400640 1 Processor -- start 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.495+0000 7f5e12400640 1 -- start start 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c0d1ac0 0x7f5e0c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e0c162020 0x7f5e0c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0cf880 con 0x7f5e0c0cf110 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c0b7b30 con 0x7f5e0c162020 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e12400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e0c0d2b90 con 0x7f5e0c0d1ac0 2024-04-12T19:50:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e11000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e11000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35426/0 (socket says 172.21.15.32:35426) 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.496+0000 7f5e11000640 1 -- 172.21.15.32:0/2187486934 learned_addr learned my addr 172.21.15.32:0/2187486934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e0be00640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c0d1ac0 0x7f5e0c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 -- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c0d1ac0 msgr2=0x7f5e0c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c0d1ac0 0x7f5e0c1515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 -- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e0c162020 msgr2=0x7f5e0c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e0c162020 0x7f5e0c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 -- 172.21.15.32:0/2187486934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c152910 con 0x7f5e0c0cf110 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e11000640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc007ef0 tx=0x7f5dfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc014620 con 0x7f5e0c0cf110 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc019070 con 0x7f5e0c0cf110 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc014a00 con 0x7f5e0c0cf110 2024-04-12T19:50:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c152070 con 0x7f5e0c0cf110 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.497+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c152630 con 0x7f5e0c0cf110 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.498+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c0755c0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.499+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dfc014ba0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.500+0000 7f5e08c00640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 0x7f5de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.500+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5dfc08c4d0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.501+0000 7f5e0be00640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 0x7f5de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.501+0000 7f5e0be00640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 0x7f5de806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5df4036000 tx=0x7f5df4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.505+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dfc05a8f0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.751+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e0c0b7650 con 0x7f5e0c0cf110 2024-04-12T19:50:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.752+0000 7f5e08c00640 1 -- 172.21.15.32:0/2187486934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f5dfc05a1f0 con 0x7f5e0c0cf110 2024-04-12T19:50:41.752 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 msgr2=0x7f5de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 0x7f5de806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5df4036000 tx=0x7f5df4008030 comp rx=0 tx=0).stop 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 msgr2=0x7f5e0c14ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc007ef0 tx=0x7f5dfc00e4b0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 shutdown_connections 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5de806c8f0 0x7f5de806edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e0c162020 0x7f5e0c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e0c0d1ac0 0x7f5e0c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.754+0000 7f5e12400640 1 --2- 172.21.15.32:0/2187486934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e0c0cf110 0x7f5e0c14ed40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.755+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 >> 172.21.15.32:0/2187486934 conn(0x7f5e0c06cf00 msgr2=0x7f5e0c0b94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.755+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 shutdown_connections 2024-04-12T19:50:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41.755+0000 7f5e12400640 1 -- 172.21.15.32:0/2187486934 wait complete. 2024-04-12T19:50:41.772 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T19:50:41.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T19:50:41.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T19:50:41.773 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-12T19:50:41.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:41.773 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:41.823 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42540sec preferred_lft 42540sec 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:41.824 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:41.825 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:41.825 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-12T19:50:41.826 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-04-12T19:50:41.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T19:50:41.827 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:42.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:42.378 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T19:50:42.460 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-12T19:50:42.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:42.464 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-12T19:50:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-12T19:50:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T19:50:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-12T19:50:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-12T19:50:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-12T19:50:42.528 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:42.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:43.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:43.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:43.388 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:50:43.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:43.443 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-12T19:50:43.494 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:43.494 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:43.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T19:50:43.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-12T19:50:43.494 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:43.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:43.706 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-12T19:50:43.706 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-12T19:50:43.707 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:44.084 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:44.084 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:44.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-12T19:50:44.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T19:50:44.084 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:44.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:44.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:44.321 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-12T19:50:44.369 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-12T19:50:44.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:44.369 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-12T19:50:44.417 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-12T19:50:44.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:44.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:50:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:50:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:50:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:50:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:50:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:45.291 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:45.331 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:50:45.331 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:50:45.331 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:45.331 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:45.332 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:45.332 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-12T19:50:45.332 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-12T19:50:45.332 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47426077-081d-47b8-bfde-ce036e9b4cf8" will pass to kernel 2024-04-12T19:50:45.339 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-12T19:50:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-12T19:50:45.339 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-12T19:50:45.340 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-12T19:50:45.340 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-12T19:50:45.345 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-12T19:50:45.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:45.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-12T19:50:45.448 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:45.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:45.607 INFO:teuthology.orchestra.run.smithi032.stdout:client4537 2024-04-12T19:50:45.607 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:50:45.607 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:50:45.664 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:45.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:45.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:45.816 INFO:teuthology.orchestra.run.smithi032.stdout:client4537 2024-04-12T19:50:45.816 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:45.968 INFO:teuthology.orchestra.run.smithi032.stdout:client4537 2024-04-12T19:50:45.968 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/47426077-081d-47b8-bfde-ce036e9b4cf8.client4537/status') 2024-04-12T19:50:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:46.143 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4537 (3)172.21.15.32:0/1013430150 2024-04-12T19:50:46.143 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-04-12T19:50:46.143 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-04-12T19:50:46.143 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-04-12T19:50:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 9.7142e-05 s, 721 kB/s 2024-04-12T19:50:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:46.159 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-12T19:50:46.159 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T19:50:46.159 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-12T19:50:46.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-12T19:50:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.371+0000 7ff9fb200640 1 Processor -- start 2024-04-12T19:50:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.372+0000 7ff9fb200640 1 -- start start 2024-04-12T19:50:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.373+0000 7ff9fb200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9f4162020 0x7ff9f4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.373+0000 7ff9fb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f40d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.373+0000 7ff9fb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f40d0090 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.373+0000 7ff9fb200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f40d1f50 con 0x7ff9f4162020 2024-04-12T19:50:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.373+0000 7ff9fb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f4163440 con 0x7ff9f40cf110 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.374+0000 7ff9fa800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9f40d1ac0 0x7ff9f40b62a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35420/0 (socket says 172.21.15.32:35420) 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.374+0000 7ff9fa800640 1 -- 172.21.15.32:0/1925864876 learned_addr learned my addr 172.21.15.32:0/1925864876 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.374+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/1925864876 >> v2:172.21.15.32:3300/0 conn(0x7ff9f4162020 0x7ff9f4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.374+0000 7ff9f9400640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f40d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.374+0000 7ff9f8a00640 1 -- 172.21.15.32:0/1925864876 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2101068117 0 0) 0x7ff9f0001490 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.375+0000 7ff9f8a00640 1 -- 172.21.15.32:0/1925864876 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9dc003520 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.375+0000 7ff9f9e00640 1 -- 172.21.15.32:0/1925864876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 msgr2=0x7ff9f40d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.375+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f40d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.375+0000 7ff9f9e00640 1 -- 172.21.15.32:0/1925864876 >> v1:172.21.15.32:6789/0 conn(0x7ff9f40d1ac0 legacy=0x7ff9f40b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.375+0000 7ff9f9e00640 1 -- 172.21.15.32:0/1925864876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f40cfad0 con 0x7ff9f4162020 2024-04-12T19:50:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.376+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f4162020 0x7ff9f4162f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097a0 tx=0x7ff9e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcebb68e7ab54c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.376+0000 7ff9f8a00640 1 -- 172.21.15.32:0/1925864876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8035020 con 0x7ff9f4162020 2024-04-12T19:50:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.376+0000 7ff9f8a00640 1 -- 172.21.15.32:0/1925864876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8031830 con 0x7ff9f4162020 2024-04-12T19:50:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.376+0000 7ff9f8a00640 1 -- 172.21.15.32:0/1925864876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8031c00 con 0x7ff9f4162020 2024-04-12T19:50:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 -- 172.21.15.32:0/1925864876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f4162020 msgr2=0x7ff9f4162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f4162020 0x7ff9f4162f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097a0 tx=0x7ff9e802b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 -- 172.21.15.32:0/1925864876 shutdown_connections 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f40d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 --2- 172.21.15.32:0/1925864876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f4162020 0x7ff9f4162f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 -- 172.21.15.32:0/1925864876 >> 172.21.15.32:0/1925864876 conn(0x7ff9f406cf00 msgr2=0x7ff9f406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 -- 172.21.15.32:0/1925864876 shutdown_connections 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.377+0000 7ff9fb200640 1 -- 172.21.15.32:0/1925864876 wait complete. 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.379+0000 7ff9fb200640 1 Processor -- start 2024-04-12T19:50:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.379+0000 7ff9fb200640 1 -- start start 2024-04-12T19:50:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f41e9860 0x7ff9f415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f40cf880 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f40b7b30 con 0x7ff9f41e9860 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.380+0000 7ff9fb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f40d2b90 con 0x7ff9f40cf110 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35298/0 (socket says 172.21.15.32:35298) 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 -- 172.21.15.32:0/336958480 learned_addr learned my addr 172.21.15.32:0/336958480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9fa800640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f41e9860 0x7ff9f415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 -- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 msgr2=0x7ff9f4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 -- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f41e9860 msgr2=0x7ff9f415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f41e9860 0x7ff9f415afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 -- 172.21.15.32:0/336958480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f41e9e90 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.381+0000 7ff9f9400640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e870 tx=0x7ff9e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.382+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4017070 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.382+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e400c940 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.382+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400cd30 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.382+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f41ea170 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.382+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41ea730 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.383+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f40755c0 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.384+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9e401a070 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.385+0000 7ff9ee000640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.385+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e408bdf0 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.385+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.386+0000 7ff9f9e00640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 0x7ff9d806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8036000 tx=0x7ff9e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.390+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9e405a180 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.633+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff9f40d25a0 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.634+0000 7ff9ee000640 1 -- 172.21.15.32:0/336958480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff9e4059a80 con 0x7ff9f40d1ac0 2024-04-12T19:50:46.634 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-12T19:50:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 msgr2=0x7ff9d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 0x7ff9d806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8036000 tx=0x7ff9e8008030 comp rx=0 tx=0).stop 2024-04-12T19:50:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 msgr2=0x7ff9f41e9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e870 tx=0x7ff9e400ed30 comp rx=0 tx=0).stop 2024-04-12T19:50:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 shutdown_connections 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f41e9860 0x7ff9f415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9f40d1ac0 0x7ff9f41e9320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 --2- 172.21.15.32:0/336958480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f40cf110 0x7ff9f4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 >> 172.21.15.32:0/336958480 conn(0x7ff9f406cf00 msgr2=0x7ff9f40cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 shutdown_connections 2024-04-12T19:50:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:46.637+0000 7ff9fb200640 1 -- 172.21.15.32:0/336958480 wait complete. 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T19:50:46.656 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T19:50:46.657 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T19:50:46.657 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-12T19:50:46.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:46.657 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-12T19:50:46.703 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:46.704 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42535sec preferred_lft 42535sec 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether a6:fb:7c:29:2e:b6 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-12T19:50:46.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::a47c:8fff:feea:b793/64 scope link 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether a6:fb:7c:29:2e:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::a4fb:7cff:fe29:2eb6/64 scope link 2024-04-12T19:50:46.706 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:46.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:46.707 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T19:50:46.851 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-12T19:50:46.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:46.856 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-12T19:50:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-12T19:50:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T19:50:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-12T19:50:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-12T19:50:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T19:50:46.916 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:46.917 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:46.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-12T19:50:46.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-12T19:50:46.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-12T19:50:46.917 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:47.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:47.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:47.183 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:50:47.235 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-12T19:50:47.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:47.236 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-12T19:50:47.293 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-12T19:50:47.297 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:47.297 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:47.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-12T19:50:47.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-12T19:50:47.297 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:47.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:47.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ba:52:bb:1e:a2:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b852:bbff:fe1e:a202/64 scope link 2024-04-12T19:50:47.616 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:47.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-12T19:50:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ba:52:bb:1e:a2:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b852:bbff:fe1e:a202/64 scope link 2024-04-12T19:50:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:47.729 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-12T19:50:47.729 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:47.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:48.085 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:48.085 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:50:48.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2024-04-12T19:50:48.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2024-04-12T19:50:48.085 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:50:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:48.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:48.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:48.332 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-12T19:50:48.388 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-12T19:50:48.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:48.388 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-12T19:50:48.442 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-12T19:50:48.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:48.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-12T19:50:48.589 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-12T19:50:48.589 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-12T19:50:48.589 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:48.612 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-12T19:50:48.612 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-12T19:50:48.612 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:48.613 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:48.641 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-12T19:50:48.641 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-12T19:50:48.642 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=47426077-081d-47b8-bfde-ce036e9b4cf8" will pass to kernel 2024-04-12T19:50:48.643 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-12T19:50:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-12T19:50:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-12T19:50:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-12T19:50:48.643 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-12T19:50:48.649 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-12T19:50:48.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:48.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-12T19:50:48.737 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:48.860 INFO:teuthology.orchestra.run.smithi032.stdout:client4415 2024-04-12T19:50:48.861 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:50:48.861 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:50:48.910 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:49.036 INFO:teuthology.orchestra.run.smithi032.stdout:client4415 2024-04-12T19:50:49.036 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:49.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:49.160 INFO:teuthology.orchestra.run.smithi032.stdout:client4415 2024-04-12T19:50:49.160 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/47426077-081d-47b8-bfde-ce036e9b4cf8.client4415/status') 2024-04-12T19:50:49.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:49.310 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4415 (3)172.21.15.32:0/1968236093 2024-04-12T19:50:49.311 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-04-12T19:50:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-04-12T19:50:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-04-12T19:50:49.311 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 6.7086e-05 s, 1.0 MB/s 2024-04-12T19:50:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:49.324 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-12T19:50:49.325 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T19:50:49.325 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T19:50:49.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-12T19:50:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.550+0000 7f8367200640 1 Processor -- start 2024-04-12T19:50:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.551+0000 7f8367200640 1 -- start start 2024-04-12T19:50:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8367200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f8368162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8367200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83680d0090 con 0x7f83680d1c90 2024-04-12T19:50:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8367200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83680cfad0 con 0x7f83680cf110 2024-04-12T19:50:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8367200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8368163440 con 0x7f8368162020 2024-04-12T19:50:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8366800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83680d1c90 0x7f83680d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35428/0 (socket says 172.21.15.32:35428) 2024-04-12T19:50:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.552+0000 7f8366800640 1 -- 172.21.15.32:0/133513524 learned_addr learned my addr 172.21.15.32:0/133513524 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365e00640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f8368162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219828501 0 0) 0x7f8360001490 con 0x7f83680d1c90 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f834c003520 con 0x7f83680d1c90 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2034265140 0 0) 0x7f834c003520 con 0x7f83680d1c90 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 msgr2=0x7f8368162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f8368162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 >> v1:172.21.15.169:6789/0 conn(0x7f83680cf110 legacy=0x7f83680d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.553+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83680cfdb0 con 0x7f83680d1c90 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f8360001bc0 con 0x7f83680d1c90 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 >> v1:172.21.15.32:6789/0 conn(0x7f83680d1c90 legacy=0x7f83680d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f834c0039e0 0x7f834c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f834c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c004d40 0x7f834c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8368163440 con 0x7f834c004d40 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f834c00ba70 con 0x7f834c0039e0 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f834c00bc70 con 0x7f8368162020 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c004d40 0x7f834c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8365e00640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f834c0039e0 0x7f834c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f835fe00640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f834c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 -- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 msgr2=0x7f834c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8368162020 0x7f834c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 -- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f834c0039e0 msgr2=0x7f834c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f834c0039e0 0x7f834c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.554+0000 7f8366800640 1 -- 172.21.15.32:0/133513524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c003520 con 0x7f834c004d40 2024-04-12T19:50:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.556+0000 7f8366800640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c004d40 0x7f834c0051c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f836000aa20 tx=0x7f834c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9171ef8f5be00e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.556+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360038070 con 0x7f834c004d40 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.556+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8360033620 con 0x7f834c004d40 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.556+0000 7f8365400640 1 -- 172.21.15.32:0/133513524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836000a670 con 0x7f834c004d40 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.556+0000 7f8367200640 1 -- 172.21.15.32:0/133513524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c004d40 msgr2=0x7f834c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8367200640 1 --2- 172.21.15.32:0/133513524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c004d40 0x7f834c0051c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f836000aa20 tx=0x7f834c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8366800640 1 -- 172.21.15.32:0/133513524 reap_dead start 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8367200640 1 -- 172.21.15.32:0/133513524 shutdown_connections 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8367200640 1 -- 172.21.15.32:0/133513524 >> 172.21.15.32:0/133513524 conn(0x7f836806cf00 msgr2=0x7f836806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8367200640 1 -- 172.21.15.32:0/133513524 shutdown_connections 2024-04-12T19:50:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.557+0000 7f8367200640 1 -- 172.21.15.32:0/133513524 wait complete. 2024-04-12T19:50:49.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.559+0000 7f8367200640 1 Processor -- start 2024-04-12T19:50:49.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.559+0000 7f8367200640 1 -- start start 2024-04-12T19:50:49.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f834c004d40 0x7f836815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 0x7f83681f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83680cf880 con 0x7f836815f8a0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83680b7b30 con 0x7f8368160ed0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8367200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83680d2e70 con 0x7f834c004d40 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35314/0 (socket says 172.21.15.32:35314) 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 -- 172.21.15.32:0/1066322511 learned_addr learned my addr 172.21.15.32:0/1066322511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8365e00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f834c004d40 0x7f836815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8366800640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 0x7f83681f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 -- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f834c004d40 msgr2=0x7f836815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f834c004d40 0x7f836815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 -- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 msgr2=0x7f83681f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 0x7f83681f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f835fe00640 1 -- 172.21.15.32:0/1066322511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836815fe30 con 0x7f836815f8a0 2024-04-12T19:50:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.560+0000 7f8366800640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 0x7f83681f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f835fe00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f835000e580 tx=0x7f835000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500052e0 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350018070 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350003810 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8368160110 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.561+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681606d0 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.562+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83680755c0 con 0x7f836815f8a0 2024-04-12T19:50:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.563+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83500039b0 con 0x7f836815f8a0 2024-04-12T19:50:49.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.564+0000 7f835e000640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:49.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.564+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f835008b8b0 con 0x7f836815f8a0 2024-04-12T19:50:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.565+0000 7f8365e00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.565+0000 7f8365e00640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 0x7f834406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8368159ff0 tx=0x7f8358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.569+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8350059c50 con 0x7f836815f8a0 2024-04-12T19:50:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.822+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83680b7650 con 0x7f836815f8a0 2024-04-12T19:50:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.824+0000 7f835e000640 1 -- 172.21.15.32:0/1066322511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8350059550 con 0x7f836815f8a0 2024-04-12T19:50:49.824 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 msgr2=0x7f834406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 0x7f834406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8368159ff0 tx=0x7f8358008040 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 msgr2=0x7f836814ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f835000e580 tx=0x7f835000ea40 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 shutdown_connections 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8368160ed0 0x7f83681f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f836815f8a0 0x7f836814ca80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 --2- 172.21.15.32:0/1066322511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f834c004d40 0x7f836815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.826+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 >> 172.21.15.32:0/1066322511 conn(0x7f836806cf00 msgr2=0x7f83680cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.827+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 shutdown_connections 2024-04-12T19:50:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:49.827+0000 7f8367200640 1 -- 172.21.15.32:0/1066322511 wait complete. 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T19:50:49.846 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-12T19:50:49.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:49.847 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-12T19:50:49.903 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-12T19:50:49.904 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-12T19:50:49.905 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:49.906 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T19:50:49.906 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:49.906 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:49.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-04-12T19:50:49.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-04-12T19:50:49.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-04-12T19:50:49.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T19:50:49.907 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:50.013 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:50.431 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:50.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:50.441 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T19:50:50.515 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-12T19:50:50.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:50.519 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-12T19:50:50.575 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-12T19:50:50.575 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T19:50:50.575 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-12T19:50:50.575 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-12T19:50:50.576 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-12T19:50:50.577 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:50.680 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:51.377 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:51.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:51.387 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:50:51.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:51.441 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-12T19:50:51.490 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:51.490 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:51.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T19:50:51.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-12T19:50:51.490 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:51.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:51.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:51.699 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-12T19:50:51.699 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:51.805 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:52.035 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:52.045 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:52.045 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:52.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-04-12T19:50:52.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T19:50:52.046 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:52.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:52.261 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:52.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:52.270 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-12T19:50:52.312 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-12T19:50:52.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:52.313 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-12T19:50:52.402 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-12T19:50:52.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:52.403 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-12T19:50:52.537 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-12T19:50:52.537 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-12T19:50:52.537 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:53.247 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-12T19:50:53.247 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-12T19:50:53.247 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:53.247 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:53.247 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-12T19:50:53.280 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=47426077-081d-47b8-bfde-ce036e9b4cf8" will pass to kernel 2024-04-12T19:50:53.284 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-12T19:50:53.285 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2024-04-12T19:50:53.285 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-12T19:50:53.285 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2024-04-12T19:50:53.285 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-12T19:50:53.290 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-12T19:50:53.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:53.290 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-12T19:50:53.382 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:53.491 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:53.496 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:53.506 INFO:teuthology.orchestra.run.smithi169.stdout:client4543 2024-04-12T19:50:53.506 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:50:53.506 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:50:53.555 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:53.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:53.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:53.722 INFO:teuthology.orchestra.run.smithi169.stdout:client4543 2024-04-12T19:50:53.722 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:53.825 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:53.830 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:53.840 INFO:teuthology.orchestra.run.smithi169.stdout:client4543 2024-04-12T19:50:53.840 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/47426077-081d-47b8-bfde-ce036e9b4cf8.client4543/status') 2024-04-12T19:50:53.942 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:53.983 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4543 (3)192.168.0.1:0/84487692 2024-04-12T19:50:53.983 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-12T19:50:53.983 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-12T19:50:53.983 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-12T19:50:53.983 INFO:teuthology.orchestra.run.smithi169.stderr:67 bytes copied, 8.6452e-05 s, 775 kB/s 2024-04-12T19:50:53.986 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:53.996 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-12T19:50:53.996 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T19:50:53.996 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-12T19:50:53.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-12T19:50:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.229+0000 7fb9d2400640 1 Processor -- start 2024-04-12T19:50:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.229+0000 7fb9d2400640 1 -- start start 2024-04-12T19:50:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.230+0000 7fb9d2400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9cc0cfc50 0x7fb9cc0d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9d2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9d2400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9cc15f200 con 0x7fb9cc15ee20 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9d2400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9cc0d46e0 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9d2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9cc0d21f0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9c3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9cc0cfc50 0x7fb9cc0d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:35918/0 (socket says 172.21.15.32:35918) 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9cbe00640 1 -- 172.21.15.32:0/2383594506 learned_addr learned my addr 172.21.15.32:0/2383594506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.231+0000 7fb9cb400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb9cc15ee20 0x7fb9cc0d3fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:55178/0 (socket says 172.21.15.32:55178) 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.232+0000 7fb9c3e00640 1 -- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 msgr2=0x7fb9cc0b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.232+0000 7fb9c3e00640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.232+0000 7fb9c3e00640 1 -- 172.21.15.32:0/2383594506 >> v1:172.21.15.169:6789/0 conn(0x7fb9cc15ee20 legacy=0x7fb9cc0d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.232+0000 7fb9c3e00640 1 -- 172.21.15.32:0/2383594506 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc0b6500 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.232+0000 7fb9cbe00640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:50:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.234+0000 7fb9c3e00640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc0d1840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097a0 tx=0x7fb9b402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f50465b014311b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:54.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.235+0000 7fb9caa00640 1 -- 172.21.15.32:0/2383594506 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4039070 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.235+0000 7fb9caa00640 1 -- 172.21.15.32:0/2383594506 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b40316f0 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.235+0000 7fb9caa00640 1 -- 172.21.15.32:0/2383594506 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4031ac0 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 -- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 msgr2=0x7fb9cc0d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc0d1840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097a0 tx=0x7fb9b402b790 comp rx=0 tx=0).stop 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 -- 172.21.15.32:0/2383594506 shutdown_connections 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 --2- 172.21.15.32:0/2383594506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc0d1840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 -- 172.21.15.32:0/2383594506 >> 172.21.15.32:0/2383594506 conn(0x7fb9cc06d490 msgr2=0x7fb9cc06d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:54.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 -- 172.21.15.32:0/2383594506 shutdown_connections 2024-04-12T19:50:54.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.236+0000 7fb9d2400640 1 -- 172.21.15.32:0/2383594506 wait complete. 2024-04-12T19:50:54.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.237+0000 7fb9d2400640 1 Processor -- start 2024-04-12T19:50:54.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.238+0000 7fb9d2400640 1 -- start start 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.238+0000 7fb9d2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9d2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9d2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc15ee20 0x7fb9cc0b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9d2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc0b6da0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9d2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc0b6ac0 con 0x7fb9cc0cfc50 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9d2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9cc0b7080 con 0x7fb9cc15ee20 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53074/0 (socket says 172.21.15.32:53074) 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9c3e00640 1 -- 172.21.15.32:0/332450072 learned_addr learned my addr 172.21.15.32:0/332450072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9cb400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9cbe00640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc15ee20 0x7fb9cc0b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9c3e00640 1 -- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc15ee20 msgr2=0x7fb9cc0b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.239+0000 7fb9c3e00640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc15ee20 0x7fb9cc0b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3e00640 1 -- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 msgr2=0x7fb9cc161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3e00640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc161c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3e00640 1 -- 172.21.15.32:0/332450072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc1f8a90 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3e00640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4037040 tx=0x7fb9b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4043070 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b40346a0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4039040 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc1f5ad0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.240+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1f57d0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.242+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc068c80 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.243+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9b4046070 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.244+0000 7fb9c3400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.244+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb9b40ad670 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.245+0000 7fb9cb400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.245+0000 7fb9cb400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 0x7fb9a806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0ac980 tx=0x7fb9bc007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:54.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.250+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9b407ba90 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.501+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb9cc0b57f0 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.502+0000 7fb9c3400640 1 -- 172.21.15.32:0/332450072 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb9b407b390 con 0x7fb9cc0d1d80 2024-04-12T19:50:54.501 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 msgr2=0x7fb9a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 0x7fb9a806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0ac980 tx=0x7fb9bc007ce0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 msgr2=0x7fb9cc0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4037040 tx=0x7fb9b40046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 shutdown_connections 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb9a806c8f0 0x7fb9a806edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9cc15ee20 0x7fb9cc0b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9cc0d1d80 0x7fb9cc0b8c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 --2- 172.21.15.32:0/332450072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9cc0cfc50 0x7fb9cc161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 >> 172.21.15.32:0/332450072 conn(0x7fb9cc06d490 msgr2=0x7fb9cc0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 shutdown_connections 2024-04-12T19:50:54.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:54.505+0000 7fb9d2400640 1 -- 172.21.15.32:0/332450072 wait complete. 2024-04-12T19:50:54.523 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T19:50:54.524 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-12T19:50:54.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:54.525 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-12T19:50:54.583 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42525sec preferred_lft 42525sec 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.585 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether aa:cf:4c:2e:85:77 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::1053:24ff:fe24:39c2/64 scope link 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether aa:cf:4c:2e:85:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::a8cf:4cff:fe2e:8577/64 scope link 2024-04-12T19:50:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:54.587 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T19:50:54.681 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-12T19:50:54.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:54.685 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-12T19:50:54.741 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-12T19:50:54.742 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T19:50:54.742 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-12T19:50:54.742 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-12T19:50:54.742 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T19:50:54.742 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:54.742 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:54.742 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-12T19:50:54.742 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-12T19:50:54.743 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-12T19:50:54.743 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:54.850 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:54.985 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:54.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:54.995 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:50:55.043 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-12T19:50:55.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:55.043 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-12T19:50:55.090 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-12T19:50:55.091 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:55.091 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:55.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-12T19:50:55.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-12T19:50:55.091 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:55.193 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:55.286 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:55.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:55.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-12T19:50:55.383 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 46:92:32:b6:94:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::4492:32ff:feb6:9410/64 scope link 2024-04-12T19:50:55.384 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:55.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-12T19:50:55.482 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:50:55.482 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:50:55.482 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:50:55.482 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 46:92:32:b6:94:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::4492:32ff:feb6:9410/64 scope link 2024-04-12T19:50:55.483 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:50:55.486 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-12T19:50:55.487 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:55.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:55.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:55.818 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:50:55.818 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-12T19:50:55.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2024-04-12T19:50:55.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2024-04-12T19:50:55.818 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-12T19:50:55.918 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:56.035 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:56.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:56.044 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-12T19:50:56.092 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-12T19:50:56.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:50:56.092 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-12T19:50:56.139 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-12T19:50:56.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:56.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-12T19:50:56.275 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-12T19:50:56.276 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-12T19:50:56.276 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-12T19:50:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-12T19:50:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:56.328 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-12T19:50:56.328 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-12T19:50:56.328 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2024-04-12T19:50:56.328 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:50:56.328 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:50:56.329 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-12T19:50:56.329 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-12T19:50:56.329 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=47426077-081d-47b8-bfde-ce036e9b4cf8" will pass to kernel 2024-04-12T19:50:56.330 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-12T19:50:56.330 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2024-04-12T19:50:56.330 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-12T19:50:56.330 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2024-04-12T19:50:56.330 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-12T19:50:56.334 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-12T19:50:56.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:50:56.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-12T19:50:56.415 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:56.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:56.520 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:56.529 INFO:teuthology.orchestra.run.smithi169.stdout:client4635 2024-04-12T19:50:56.529 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:50:56.529 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:50:56.581 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:56.689 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:56.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:56.703 INFO:teuthology.orchestra.run.smithi169.stdout:client4635 2024-04-12T19:50:56.704 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:50:56.806 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:50:56.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:56.821 INFO:teuthology.orchestra.run.smithi169.stdout:client4635 2024-04-12T19:50:56.822 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/47426077-081d-47b8-bfde-ce036e9b4cf8.client4635/status') 2024-04-12T19:50:56.919 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:50:56.958 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4635 (3)172.21.15.169:0/3133426046 2024-04-12T19:50:56.959 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-12T19:50:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-12T19:50:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-12T19:50:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:71 bytes copied, 7.8485e-05 s, 905 kB/s 2024-04-12T19:50:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-12T19:50:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:50:56.971 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-12T19:50:56.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.219+0000 7f2e0de00640 1 Processor -- start 2024-04-12T19:50:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.219+0000 7f2e0de00640 1 -- start start 2024-04-12T19:50:57.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.221+0000 7f2e0de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e080cefc0 0x7f2e080b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.221+0000 7f2e0de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e080b3e10 0x7f2e0815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.221+0000 7f2e0de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e080b4be0 con 0x7f2e081461f0 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.221+0000 7f2e0de00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e080a9640 con 0x7f2e080b3e10 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.221+0000 7f2e0de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2e080b4620 con 0x7f2e080cefc0 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e080b3e10 0x7f2e0815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e080b3e10 0x7f2e0815d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53076/0 (socket says 172.21.15.32:53076) 2024-04-12T19:50:57.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 -- 172.21.15.32:0/2572900997 learned_addr learned my addr 172.21.15.32:0/2572900997 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e06000640 1 --2- 172.21.15.32:0/2572900997 >> v2:172.21.15.169:3300/0 conn(0x7f2e080cefc0 0x7f2e080b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 -- 172.21.15.32:0/2572900997 >> v2:172.21.15.169:3300/0 conn(0x7f2e080cefc0 msgr2=0x7f2e080b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 --2- 172.21.15.32:0/2572900997 >> v2:172.21.15.169:3300/0 conn(0x7f2e080cefc0 0x7f2e080b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 -- 172.21.15.32:0/2572900997 >> v1:172.21.15.169:6789/0 conn(0x7f2e081461f0 legacy=0x7f2e080ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:57.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.222+0000 7f2e07400640 1 -- 172.21.15.32:0/2572900997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e080b4900 con 0x7f2e080b3e10 2024-04-12T19:50:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.223+0000 7f2e07400640 1 --2- 172.21.15.32:0/2572900997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e0815d7b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e08073560 tx=0x7f2e0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6675a40dd2a5de server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.224+0000 7f2e05600640 1 -- 172.21.15.32:0/2572900997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00035020 con 0x7f2e080b3e10 2024-04-12T19:50:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.224+0000 7f2e05600640 1 -- 172.21.15.32:0/2572900997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e00031870 con 0x7f2e080b3e10 2024-04-12T19:50:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.224+0000 7f2e05600640 1 -- 172.21.15.32:0/2572900997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00031c40 con 0x7f2e080b3e10 2024-04-12T19:50:57.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.224+0000 7f2e0de00640 1 -- 172.21.15.32:0/2572900997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 msgr2=0x7f2e0815d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.224+0000 7f2e0de00640 1 --2- 172.21.15.32:0/2572900997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e0815d7b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e08073560 tx=0x7f2e0002b910 comp rx=0 tx=0).stop 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 -- 172.21.15.32:0/2572900997 shutdown_connections 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 --2- 172.21.15.32:0/2572900997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e0815d7b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 --2- 172.21.15.32:0/2572900997 >> v2:172.21.15.169:3300/0 conn(0x7f2e080cefc0 0x7f2e080b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 -- 172.21.15.32:0/2572900997 >> 172.21.15.32:0/2572900997 conn(0x7f2e080b27b0 msgr2=0x7f2e080b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 -- 172.21.15.32:0/2572900997 shutdown_connections 2024-04-12T19:50:57.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.225+0000 7f2e0de00640 1 -- 172.21.15.32:0/2572900997 wait complete. 2024-04-12T19:50:57.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.227+0000 7f2e0de00640 1 Processor -- start 2024-04-12T19:50:57.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.227+0000 7f2e0de00640 1 -- start start 2024-04-12T19:50:57.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e080cefc0 0x7f2e081416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e081461f0 0x7f2e080c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081465d0 con 0x7f2e080b3e10 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815d290 con 0x7f2e080cefc0 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e0de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e080b4280 con 0x7f2e081461f0 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53080/0 (socket says 172.21.15.32:53080) 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 -- 172.21.15.32:0/3226483970 learned_addr learned my addr 172.21.15.32:0/3226483970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 -- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e081461f0 msgr2=0x7f2e080c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e081461f0 0x7f2e080c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 -- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e080cefc0 msgr2=0x7f2e081416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e080cefc0 0x7f2e081416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.228+0000 7f2e06a00640 1 -- 172.21.15.32:0/3226483970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e080c60d0 con 0x7f2e080b3e10 2024-04-12T19:50:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2e06a00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2df000b3f0 tx=0x7f2df000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0015020 con 0x7f2e080b3e10 2024-04-12T19:50:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df0011830 con 0x7f2e080b3e10 2024-04-12T19:50:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e080c72f0 con 0x7f2e080b3e10 2024-04-12T19:50:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0011c20 con 0x7f2e080b3e10 2024-04-12T19:50:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.229+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0806e240 con 0x7f2e080b3e10 2024-04-12T19:50:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.230+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08068c80 con 0x7f2e080b3e10 2024-04-12T19:50:57.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.231+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2df0018020 con 0x7f2e080b3e10 2024-04-12T19:50:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.232+0000 7f2dff400640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 0x7f2de406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.232+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2df008b710 con 0x7f2e080b3e10 2024-04-12T19:50:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.233+0000 7f2e06000640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 0x7f2de406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.233+0000 7f2e06000640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 0x7f2de406ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2df80070b0 tx=0x7f2df8007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.237+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2df001f030 con 0x7f2e080b3e10 2024-04-12T19:50:57.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.443+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e080727d0 con 0x7f2e080b3e10 2024-04-12T19:50:57.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.446+0000 7f2dff400640 1 -- 172.21.15.32:0/3226483970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7f2df00172c0 con 0x7f2e080b3e10 2024-04-12T19:50:57.446 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:57.447 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 msgr2=0x7f2de406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 0x7f2de406ede0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2df80070b0 tx=0x7f2df8007040 comp rx=0 tx=0).stop 2024-04-12T19:50:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 msgr2=0x7f2e080c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2df000b3f0 tx=0x7f2df000b8b0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 shutdown_connections 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2de406c920 0x7f2de406ede0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e081461f0 0x7f2e080c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e080cefc0 0x7f2e081416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 --2- 172.21.15.32:0/3226483970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080b3e10 0x7f2e080c5b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 >> 172.21.15.32:0/3226483970 conn(0x7f2e080b27b0 msgr2=0x7f2e080b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.449+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 shutdown_connections 2024-04-12T19:50:57.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.450+0000 7f2e0de00640 1 -- 172.21.15.32:0/3226483970 wait complete. 2024-04-12T19:50:57.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:57.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.704+0000 7fcd1c400640 1 Processor -- start 2024-04-12T19:50:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.704+0000 7fcd1c400640 1 -- start start 2024-04-12T19:50:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd1415ecc0 0x7fcd140d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1c400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd140ba4d0 con 0x7fcd140b3720 2024-04-12T19:50:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd140ba7b0 con 0x7fcd140d3270 2024-04-12T19:50:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1415f0a0 con 0x7fcd1415ecc0 2024-04-12T19:50:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1a400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcd140b3720 0x7fcd140cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37368/0 (socket says 172.21.15.32:37368) 2024-04-12T19:50:57.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.705+0000 7fcd1a400640 1 -- 172.21.15.32:0/2720558550 learned_addr learned my addr 172.21.15.32:0/2720558550 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd19a00640 1 --2- 172.21.15.32:0/2720558550 >> v2:172.21.15.169:3300/0 conn(0x7fcd1415ecc0 0x7fcd140d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1683696527 0 0) 0x7fcd140ba4d0 con 0x7fcd140b3720 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccfc003520 con 0x7fcd140b3720 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2096489856 0 0) 0x7fccfc003520 con 0x7fcd140b3720 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 >> v2:172.21.15.169:3300/0 conn(0x7fcd1415ecc0 msgr2=0x7fcd140d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 --2- 172.21.15.32:0/2720558550 >> v2:172.21.15.169:3300/0 conn(0x7fcd1415ecc0 0x7fcd140d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 >> v1:172.21.15.169:6789/0 conn(0x7fcd140d3270 legacy=0x7fcd140b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140b8df0 con 0x7fcd140b3720 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fcd0c001670 con 0x7fcd140b3720 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 >> v1:172.21.15.32:6789/0 conn(0x7fcd140b3720 legacy=0x7fcd140cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccfc0039b0 0x7fccfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 0x7fccfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.706+0000 7fcd13e00640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0051c0 0x7fccfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1415f0a0 con 0x7fccfc004800 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc00c060 con 0x7fccfc0051c0 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc00c260 con 0x7fccfc0039b0 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 0x7fccfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 -- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccfc0039b0 msgr2=0x7fccfc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccfc0039b0 0x7fccfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 -- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0051c0 msgr2=0x7fccfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0051c0 0x7fccfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd19000640 1 -- 172.21.15.32:0/2720558550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc003520 con 0x7fccfc004800 2024-04-12T19:50:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.707+0000 7fcd1a400640 1 -- 172.21.15.32:0/2720558550 reap_dead start 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.708+0000 7fcd19000640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 0x7fccfc004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd00001750 tx=0x7fccfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f592dc59235aa2c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00001370 con 0x7fccfc004800 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd000052d0 con 0x7fccfc004800 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd13e00640 1 -- 172.21.15.32:0/2720558550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00003830 con 0x7fccfc004800 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 -- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 msgr2=0x7fccfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 0x7fccfc004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcd00001750 tx=0x7fccfc005830 comp rx=0 tx=0).stop 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 -- 172.21.15.32:0/2720558550 shutdown_connections 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 --2- 172.21.15.32:0/2720558550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccfc004800 0x7fccfc004c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 -- 172.21.15.32:0/2720558550 >> 172.21.15.32:0/2720558550 conn(0x7fcd14073480 msgr2=0x7fcd14073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 -- 172.21.15.32:0/2720558550 shutdown_connections 2024-04-12T19:50:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.709+0000 7fcd1c400640 1 -- 172.21.15.32:0/2720558550 wait complete. 2024-04-12T19:50:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.711+0000 7fcd1c400640 1 Processor -- start 2024-04-12T19:50:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.711+0000 7fcd1c400640 1 -- start start 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc004800 0x7fcd140c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd141561c0 0x7fcd14161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140b3c70 con 0x7fcd14161950 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140ba280 con 0x7fccfc004800 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.712+0000 7fcd1c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd140b3df0 con 0x7fcd141561c0 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53094/0 (socket says 172.21.15.32:53094) 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 -- 172.21.15.32:0/1674392214 learned_addr learned my addr 172.21.15.32:0/1674392214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd19a00640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc004800 0x7fcd140c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd19000640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd141561c0 0x7fcd14161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 -- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd141561c0 msgr2=0x7fcd14161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd141561c0 0x7fcd14161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 -- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc004800 msgr2=0x7fcd140c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc004800 0x7fcd140c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 -- 172.21.15.32:0/1674392214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14161ed0 con 0x7fcd14161950 2024-04-12T19:50:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.713+0000 7fcd1a400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd140c8300 tx=0x7fcd0c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.714+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c004bf0 con 0x7fcd14161950 2024-04-12T19:50:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.714+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c004db0 con 0x7fcd14161950 2024-04-12T19:50:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.714+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c007270 con 0x7fcd14161950 2024-04-12T19:50:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.714+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd140c86c0 con 0x7fcd14161950 2024-04-12T19:50:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.714+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd140c8c80 con 0x7fcd14161950 2024-04-12T19:50:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.715+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd140c8e80 con 0x7fcd14161950 2024-04-12T19:50:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.716+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd0c007410 con 0x7fcd14161950 2024-04-12T19:50:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.717+0000 7fcd11600640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 0x7fccf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.717+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd0c08b110 con 0x7fcd14161950 2024-04-12T19:50:57.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.717+0000 7fcd19a00640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 0x7fccf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:57.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.718+0000 7fcd19a00640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 0x7fccf406ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd1406a920 tx=0x7fcd0800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:57.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.721+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd140c8e80 con 0x7fcd14161950 2024-04-12T19:50:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.926+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd140d5730 con 0x7fcd14161950 2024-04-12T19:50:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.928+0000 7fcd11600640 1 -- 172.21.15.32:0/1674392214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fcd0c01a020 con 0x7fcd14161950 2024-04-12T19:50:57.927 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.930+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 msgr2=0x7fccf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.930+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 0x7fccf406ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd1406a920 tx=0x7fcd0800b040 comp rx=0 tx=0).stop 2024-04-12T19:50:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.930+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 msgr2=0x7fcd1415a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.930+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd140c8300 tx=0x7fcd0c00c890 comp rx=0 tx=0).stop 2024-04-12T19:50:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 shutdown_connections 2024-04-12T19:50:57.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fccf406c920 0x7fccf406ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd14161950 0x7fcd1415a830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd141561c0 0x7fcd14161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 --2- 172.21.15.32:0/1674392214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc004800 0x7fcd140c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 >> 172.21.15.32:0/1674392214 conn(0x7fcd14073480 msgr2=0x7fcd1415da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 shutdown_connections 2024-04-12T19:50:57.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:57.931+0000 7fcd1c400640 1 -- 172.21.15.32:0/1674392214 wait complete. 2024-04-12T19:50:57.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:50:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.184+0000 7efeb2600640 1 Processor -- start 2024-04-12T19:50:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.185+0000 7efeb2600640 1 -- start start 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.186+0000 7efeb2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15c680 0x7efeac162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.186+0000 7efeb2600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeac0b6c70 0x7efeac15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.186+0000 7efeb2600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac0b70d0 con 0x7efeac15fd90 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.186+0000 7efeb2600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efeac162dd0 con 0x7efeac0b6c70 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.186+0000 7efeb2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efeac1630b0 con 0x7efeac15c680 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeabe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efeac0b6c70 0x7efeac15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeb1c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efeac15fd90 0x7efeac0d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37378/0 (socket says 172.21.15.32:37378) 2024-04-12T19:50:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeb1c00640 1 -- 172.21.15.32:0/2279568751 learned_addr learned my addr 172.21.15.32:0/2279568751 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeb1200640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15c680 0x7efeac162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeabe00640 1 -- 172.21.15.32:0/2279568751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15c680 msgr2=0x7efeac162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeabe00640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15c680 0x7efeac162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeab400640 1 -- 172.21.15.32:0/2279568751 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754440368 0 0) 0x7efeac0b70d0 con 0x7efeac15fd90 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeabe00640 1 -- 172.21.15.32:0/2279568751 >> v1:172.21.15.32:6789/0 conn(0x7efeac15fd90 legacy=0x7efeac0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.187+0000 7efeabe00640 1 -- 172.21.15.32:0/2279568751 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac163390 con 0x7efeac0b6c70 2024-04-12T19:50:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.189+0000 7efeabe00640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15f850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe980097a0 tx=0x7efe9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b618a88d4189187 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.189+0000 7efeab400640 1 -- 172.21.15.32:0/2279568751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98035020 con 0x7efeac0b6c70 2024-04-12T19:50:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.189+0000 7efeab400640 1 -- 172.21.15.32:0/2279568751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98031830 con 0x7efeac0b6c70 2024-04-12T19:50:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.189+0000 7efeab400640 1 -- 172.21.15.32:0/2279568751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98031c00 con 0x7efeac0b6c70 2024-04-12T19:50:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 -- 172.21.15.32:0/2279568751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 msgr2=0x7efeac15f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15f850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe980097a0 tx=0x7efe9802b8d0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 -- 172.21.15.32:0/2279568751 shutdown_connections 2024-04-12T19:50:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15f850 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 --2- 172.21.15.32:0/2279568751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15c680 0x7efeac162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 -- 172.21.15.32:0/2279568751 >> 172.21.15.32:0/2279568751 conn(0x7efeac073480 msgr2=0x7efeac073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 -- 172.21.15.32:0/2279568751 shutdown_connections 2024-04-12T19:50:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.190+0000 7efeb2600640 1 -- 172.21.15.32:0/2279568751 wait complete. 2024-04-12T19:50:58.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.192+0000 7efeb2600640 1 Processor -- start 2024-04-12T19:50:58.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.192+0000 7efeb2600640 1 -- start start 2024-04-12T19:50:58.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac15c680 0x7efeac158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15fd90 0x7efeac0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efeac0d3260 con 0x7efeac0b6c70 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efeac15ca60 con 0x7efeac15c680 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efeac0d33e0 con 0x7efeac15fd90 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53126/0 (socket says 172.21.15.32:53126) 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 -- 172.21.15.32:0/496078699 learned_addr learned my addr 172.21.15.32:0/496078699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 -- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15fd90 msgr2=0x7efeac0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15fd90 0x7efeac0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 -- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac15c680 msgr2=0x7efeac158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac15c680 0x7efeac158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.193+0000 7efeb1200640 1 -- 172.21.15.32:0/496078699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac0c88b0 con 0x7efeac0b6c70 2024-04-12T19:50:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efeb1200640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efea000d830 tx=0x7efea000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea000b6c0 con 0x7efeac0b6c70 2024-04-12T19:50:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0019070 con 0x7efeac0b6c70 2024-04-12T19:50:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea000ba80 con 0x7efeac0b6c70 2024-04-12T19:50:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac0c8b90 con 0x7efeac0b6c70 2024-04-12T19:50:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.194+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac0c9150 con 0x7efeac0b6c70 2024-04-12T19:50:58.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.195+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac0755c0 con 0x7efeac0b6c70 2024-04-12T19:50:58.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.196+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efea001e460 con 0x7efeac0b6c70 2024-04-12T19:50:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.197+0000 7efea8c00640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 0x7efe8c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.197+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efea008c130 con 0x7efeac0b6c70 2024-04-12T19:50:58.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.198+0000 7efeabe00640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 0x7efe8c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.198+0000 7efeabe00640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 0x7efe8c06ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe98036000 tx=0x7efe98030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.202+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efea005a6d0 con 0x7efeac0b6c70 2024-04-12T19:50:58.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.405+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efeac0c9570 con 0x7efeac0b6c70 2024-04-12T19:50:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.406+0000 7efea8c00640 1 -- 172.21.15.32:0/496078699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7efea0059fd0 con 0x7efeac0b6c70 2024-04-12T19:50:58.405 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 msgr2=0x7efe8c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 0x7efe8c06ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe98036000 tx=0x7efe98030000 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 msgr2=0x7efeac15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efea000d830 tx=0x7efea000dcf0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 shutdown_connections 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efe8c06c940 0x7efe8c06ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac15fd90 0x7efeac0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac15c680 0x7efeac158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 --2- 172.21.15.32:0/496078699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efeac0b6c70 0x7efeac15b6b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.408+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 >> 172.21.15.32:0/496078699 conn(0x7efeac073480 msgr2=0x7efeac1614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.409+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 shutdown_connections 2024-04-12T19:50:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.409+0000 7efeb2600640 1 -- 172.21.15.32:0/496078699 wait complete. 2024-04-12T19:50:58.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:50:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.657+0000 7f38f6200640 1 Processor -- start 2024-04-12T19:50:58.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.657+0000 7f38f6200640 1 -- start start 2024-04-12T19:50:58.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.659+0000 7f38f6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.659+0000 7f38f6200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f00cd2d0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.659+0000 7f38f6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f00cd5b0 con 0x7f38f01461c0 2024-04-12T19:50:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.659+0000 7f38f6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38f00cf1d0 con 0x7f38f00725f0 2024-04-12T19:50:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.660+0000 7f38f5800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38f00ce9e0 0x7f38f006df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37390/0 (socket says 172.21.15.32:37390) 2024-04-12T19:50:58.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.660+0000 7f38f5800640 1 -- 172.21.15.32:0/693364353 learned_addr learned my addr 172.21.15.32:0/693364353 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:58.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.660+0000 7f38e7e00640 1 --2- 172.21.15.32:0/693364353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.660+0000 7f38efe00640 1 -- 172.21.15.32:0/693364353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866705884 0 0) 0x7f38f00cd2d0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.660+0000 7f38efe00640 1 -- 172.21.15.32:0/693364353 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38d4003520 con 0x7f38f00ce9e0 2024-04-12T19:50:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.661+0000 7f38e7e00640 1 -- 172.21.15.32:0/693364353 >> v1:172.21.15.169:6789/0 conn(0x7f38f01461c0 legacy=0x7f38f0071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.661+0000 7f38e7e00640 1 -- 172.21.15.32:0/693364353 >> v1:172.21.15.32:6789/0 conn(0x7f38f00ce9e0 legacy=0x7f38f006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.661+0000 7f38e7e00640 1 -- 172.21.15.32:0/693364353 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f00cd890 con 0x7f38f00725f0 2024-04-12T19:50:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.662+0000 7f38e7e00640 1 --2- 172.21.15.32:0/693364353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f00ce4a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f38d80097d0 tx=0x7f38d802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6ed73fae300c6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.663+0000 7f38efe00640 1 -- 172.21.15.32:0/693364353 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8034070 con 0x7f38f00725f0 2024-04-12T19:50:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.663+0000 7f38efe00640 1 -- 172.21.15.32:0/693364353 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d8030800 con 0x7f38f00725f0 2024-04-12T19:50:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.663+0000 7f38efe00640 1 -- 172.21.15.32:0/693364353 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8030bd0 con 0x7f38f00725f0 2024-04-12T19:50:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 -- 172.21.15.32:0/693364353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 msgr2=0x7f38f00ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 --2- 172.21.15.32:0/693364353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f00ce4a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f38d80097d0 tx=0x7f38d802b940 comp rx=0 tx=0).stop 2024-04-12T19:50:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 -- 172.21.15.32:0/693364353 shutdown_connections 2024-04-12T19:50:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 --2- 172.21.15.32:0/693364353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f00ce4a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 -- 172.21.15.32:0/693364353 >> 172.21.15.32:0/693364353 conn(0x7f38f006b930 msgr2=0x7f38f006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 -- 172.21.15.32:0/693364353 shutdown_connections 2024-04-12T19:50:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.664+0000 7f38f6200640 1 -- 172.21.15.32:0/693364353 wait complete. 2024-04-12T19:50:58.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.666+0000 7f38f6200640 1 Processor -- start 2024-04-12T19:50:58.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.666+0000 7f38f6200640 1 -- start start 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f01f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 0x7f38f00b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00cee70 con 0x7f38f00ce9e0 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38f00cd080 con 0x7f38f01461c0 2024-04-12T19:50:58.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38f6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38f006e630 con 0x7f38f00725f0 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.667+0000 7f38e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53140/0 (socket says 172.21.15.32:53140) 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 -- 172.21.15.32:0/3369593281 learned_addr learned my addr 172.21.15.32:0/3369593281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38f5800640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 0x7f38f00b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38f4e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f01f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 -- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 msgr2=0x7f38f01f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f01f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 -- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 msgr2=0x7f38f00b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 0x7f38f00b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 -- 172.21.15.32:0/3369593281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f00b5490 con 0x7f38f00ce9e0 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38f5800640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 0x7f38f00b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:58.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.668+0000 7f38e7e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38d802bf50 tx=0x7f38d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.669+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8034040 con 0x7f38f00ce9e0 2024-04-12T19:50:58.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.669+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d8030600 con 0x7f38f00ce9e0 2024-04-12T19:50:58.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.669+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f00b5770 con 0x7f38f00ce9e0 2024-04-12T19:50:58.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.669+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d80423f0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.669+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f015ba60 con 0x7f38f00ce9e0 2024-04-12T19:50:58.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.670+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f00755c0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.671+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38d8037070 con 0x7f38f00ce9e0 2024-04-12T19:50:58.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.672+0000 7f38ed600640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 0x7f38cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:58.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.673+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f38d80adc20 con 0x7f38f00ce9e0 2024-04-12T19:50:58.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.673+0000 7f38f4e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 0x7f38cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:58.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.673+0000 7f38f4e00640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 0x7f38cc06ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38e00012e0 tx=0x7f38e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:58.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.677+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38d807c0d0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.885+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38f0072e40 con 0x7f38f00ce9e0 2024-04-12T19:50:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.886+0000 7f38ed600640 1 -- 172.21.15.32:0/3369593281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f38d807b9d0 con 0x7f38f00ce9e0 2024-04-12T19:50:58.886 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:50:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.888+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 msgr2=0x7f38cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.888+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 0x7f38cc06ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38e00012e0 tx=0x7f38e0009040 comp rx=0 tx=0).stop 2024-04-12T19:50:58.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.888+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 msgr2=0x7f38f01f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:58.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.888+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f38d802bf50 tx=0x7f38d80046c0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 shutdown_connections 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38cc06c920 0x7f38cc06ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38f01461c0 0x7f38f00b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f00ce9e0 0x7f38f01f70c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 --2- 172.21.15.32:0/3369593281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38f00725f0 0x7f38f01f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 >> 172.21.15.32:0/3369593281 conn(0x7f38f006b930 msgr2=0x7f38f00b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:58.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 shutdown_connections 2024-04-12T19:50:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:58.889+0000 7f38f6200640 1 -- 172.21.15.32:0/3369593281 wait complete. 2024-04-12T19:50:58.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:50:59.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.115+0000 7fcc2a200640 1 Processor -- start 2024-04-12T19:50:59.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.115+0000 7fcc2a200640 1 -- start start 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.116+0000 7fcc2a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc24072660 0x7fcc2406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.116+0000 7fcc2a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc2415d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.116+0000 7fcc2a200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2415dbb0 con 0x7fcc240cee00 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.116+0000 7fcc2a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2406f4f0 con 0x7fcc24072660 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.116+0000 7fcc2a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2415d8d0 con 0x7fcc2415cbe0 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc28e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc240cee00 0x7fcc24071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37400/0 (socket says 172.21.15.32:37400) 2024-04-12T19:50:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc28e00640 1 -- 172.21.15.32:0/3240416675 learned_addr learned my addr 172.21.15.32:0/3240416675 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774790648 0 0) 0x7fcc2415dbb0 con 0x7fcc240cee00 2024-04-12T19:50:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc0c003520 con 0x7fcc240cee00 2024-04-12T19:50:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc29800640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc2415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.117+0000 7fcc23e00640 1 --2- 172.21.15.32:0/3240416675 >> v2:172.21.15.169:3300/0 conn(0x7fcc24072660 0x7fcc2406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2140381938 0 0) 0x7fcc0c003520 con 0x7fcc240cee00 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 msgr2=0x7fcc2415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc2415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 >> v2:172.21.15.169:3300/0 conn(0x7fcc24072660 msgr2=0x7fcc2406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 --2- 172.21.15.32:0/3240416675 >> v2:172.21.15.169:3300/0 conn(0x7fcc24072660 0x7fcc2406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2406dde0 con 0x7fcc240cee00 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fcc18001670 con 0x7fcc240cee00 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 >> v1:172.21.15.32:6789/0 conn(0x7fcc240cee00 legacy=0x7fcc24071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c003fe0 0x7fcc0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2415d8d0 con 0x7fcc0c004d40 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c00ba70 con 0x7fcc0c003fe0 2024-04-12T19:50:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c00bc40 con 0x7fcc2415cbe0 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.118+0000 7fcc29800640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc23e00640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c003fe0 0x7fcc0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc29800640 1 -- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 msgr2=0x7fcc0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc28e00640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc29800640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2415cbe0 0x7fcc0c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc29800640 1 -- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c003fe0 msgr2=0x7fcc0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc29800640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c003fe0 0x7fcc0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc29800640 1 -- 172.21.15.32:0/3240416675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c003520 con 0x7fcc0c004d40 2024-04-12T19:50:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.119+0000 7fcc23e00640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c003fe0 0x7fcc0c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:50:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.120+0000 7fcc29800640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c007360 tx=0x7fcc0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f75f813760c120f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.120+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c039070 con 0x7fcc0c004d40 2024-04-12T19:50:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.120+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1c02ccd0 con 0x7fcc0c004d40 2024-04-12T19:50:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.120+0000 7fcc23400640 1 -- 172.21.15.32:0/3240416675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009be0 con 0x7fcc0c004d40 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 -- 172.21.15.32:0/3240416675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc0c004d40 msgr2=0x7fcc0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 --2- 172.21.15.32:0/3240416675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c007360 tx=0x7fcc0c005830 comp rx=0 tx=0).stop 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc29800640 1 -- 172.21.15.32:0/3240416675 reap_dead start 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 -- 172.21.15.32:0/3240416675 shutdown_connections 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 -- 172.21.15.32:0/3240416675 >> 172.21.15.32:0/3240416675 conn(0x7fcc2406b930 msgr2=0x7fcc2406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 -- 172.21.15.32:0/3240416675 shutdown_connections 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.121+0000 7fcc2a200640 1 -- 172.21.15.32:0/3240416675 wait complete. 2024-04-12T19:50:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.123+0000 7fcc2a200640 1 Processor -- start 2024-04-12T19:50:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.123+0000 7fcc2a200640 1 -- start start 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c004d40 0x7fcc240ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc240b5090 0x7fcc240c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2415d070 con 0x7fcc24157100 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2406e3a0 con 0x7fcc0c004d40 2024-04-12T19:50:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc2a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2415d1f0 con 0x7fcc240b5090 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.124+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53154/0 (socket says 172.21.15.32:53154) 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 -- 172.21.15.32:0/913654308 learned_addr learned my addr 172.21.15.32:0/913654308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 -- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc240b5090 msgr2=0x7fcc240c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc240b5090 0x7fcc240c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 -- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c004d40 msgr2=0x7fcc240ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c004d40 0x7fcc240ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 -- 172.21.15.32:0/913654308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc240ae050 con 0x7fcc24157100 2024-04-12T19:50:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc23e00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000e7e0 tx=0x7fcc1000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.125+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10017070 con 0x7fcc24157100 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.126+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1000ee90 con 0x7fcc24157100 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.126+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000f040 con 0x7fcc24157100 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.126+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc240ae330 con 0x7fcc24157100 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.126+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc240ae8f0 con 0x7fcc24157100 2024-04-12T19:50:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.127+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc240755c0 con 0x7fcc24157100 2024-04-12T19:50:59.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.128+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc1001a070 con 0x7fcc24157100 2024-04-12T19:50:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.129+0000 7fcc20c00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 0x7fcc0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.129+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc1008b740 con 0x7fcc24157100 2024-04-12T19:50:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.129+0000 7fcc28e00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 0x7fcc0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.130+0000 7fcc28e00640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 0x7fcc0406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af660 tx=0x7fcc18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.135+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc10059b60 con 0x7fcc24157100 2024-04-12T19:50:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.339+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc24072230 con 0x7fcc24157100 2024-04-12T19:50:59.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.342+0000 7fcc20c00640 1 -- 172.21.15.32:0/913654308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7fcc10059460 con 0x7fcc24157100 2024-04-12T19:50:59.342 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:50:59.342 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:50:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 msgr2=0x7fcc0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 0x7fcc0406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af660 tx=0x7fcc18009040 comp rx=0 tx=0).stop 2024-04-12T19:50:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 msgr2=0x7fcc240b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000e7e0 tx=0x7fcc1000eca0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 shutdown_connections 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcc0406c8f0 0x7fcc0406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc240b5090 0x7fcc240c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc24157100 0x7fcc240b4b50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 --2- 172.21.15.32:0/913654308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc0c004d40 0x7fcc240ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 >> 172.21.15.32:0/913654308 conn(0x7fcc2406b930 msgr2=0x7fcc240b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 shutdown_connections 2024-04-12T19:50:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.345+0000 7fcc2a200640 1 -- 172.21.15.32:0/913654308 wait complete. 2024-04-12T19:50:59.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:50:59.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:50:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.591+0000 7f072da00640 1 Processor -- start 2024-04-12T19:50:59.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.591+0000 7f072da00640 1 -- start start 2024-04-12T19:50:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f072da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f072806fc80 0x7f072815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f072da00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072815d520 con 0x7f0728072150 2024-04-12T19:50:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f072da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072815d800 con 0x7f072815a440 2024-04-12T19:50:59.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f072da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f072806e280 con 0x7f072806fc80 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f0726000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f072806fc80 0x7f072815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f0727400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0728072150 0x7f072806e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37406/0 (socket says 172.21.15.32:37406) 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.593+0000 7f0727400640 1 -- 172.21.15.32:0/2909380742 learned_addr learned my addr 172.21.15.32:0/2909380742 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.594+0000 7f0726000640 1 -- 172.21.15.32:0/2909380742 >> v1:172.21.15.169:6789/0 conn(0x7f072815a440 legacy=0x7f07280cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.594+0000 7f0726000640 1 -- 172.21.15.32:0/2909380742 >> v1:172.21.15.32:6789/0 conn(0x7f0728072150 legacy=0x7f072806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.594+0000 7f0726000640 1 -- 172.21.15.32:0/2909380742 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072815dae0 con 0x7f072806fc80 2024-04-12T19:50:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.594+0000 7f0725600640 1 -- 172.21.15.32:0/2909380742 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251679575 0 0) 0x7f072815d520 con 0x7f0728072150 2024-04-12T19:50:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.595+0000 7f0726000640 1 --2- 172.21.15.32:0/2909380742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f072815cef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f072806e400 tx=0x7f071002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222814cde23bd23c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.595+0000 7f0725600640 1 -- 172.21.15.32:0/2909380742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710034070 con 0x7f072806fc80 2024-04-12T19:50:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.595+0000 7f0725600640 1 -- 172.21.15.32:0/2909380742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710030610 con 0x7f072806fc80 2024-04-12T19:50:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.595+0000 7f0725600640 1 -- 172.21.15.32:0/2909380742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100309e0 con 0x7f072806fc80 2024-04-12T19:50:59.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 -- 172.21.15.32:0/2909380742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 msgr2=0x7f072815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 --2- 172.21.15.32:0/2909380742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f072815cef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f072806e400 tx=0x7f071002b960 comp rx=0 tx=0).stop 2024-04-12T19:50:59.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 -- 172.21.15.32:0/2909380742 shutdown_connections 2024-04-12T19:50:59.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 --2- 172.21.15.32:0/2909380742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f072815cef0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 -- 172.21.15.32:0/2909380742 >> 172.21.15.32:0/2909380742 conn(0x7f07280b2460 msgr2=0x7f07280b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 -- 172.21.15.32:0/2909380742 shutdown_connections 2024-04-12T19:50:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.596+0000 7f072da00640 1 -- 172.21.15.32:0/2909380742 wait complete. 2024-04-12T19:50:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.598+0000 7f072da00640 1 Processor -- start 2024-04-12T19:50:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.598+0000 7f072da00640 1 -- start start 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0728072150 0x7f07281e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f072815a440 0x7f07281e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f072815ddc0 con 0x7f072806fc80 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f072815a820 con 0x7f0728072150 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f072da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f072806f510 con 0x7f072815a440 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f0726a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f0726a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53166/0 (socket says 172.21.15.32:53166) 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.599+0000 7f0726a00640 1 -- 172.21.15.32:0/3608027812 learned_addr learned my addr 172.21.15.32:0/3608027812 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726000640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0728072150 0x7f07281e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0727400640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f072815a440 0x7f07281e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 -- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f072815a440 msgr2=0x7f07281e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f072815a440 0x7f07281e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 -- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0728072150 msgr2=0x7f07281e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0728072150 0x7f07281e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 -- 172.21.15.32:0/3608027812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07280b3da0 con 0x7f072806fc80 2024-04-12T19:50:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f0726a00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f071800c400 tx=0x7f071800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718017070 con 0x7f072806fc80 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07180052d0 con 0x7f072806fc80 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718003830 con 0x7f072806fc80 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.600+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07280b4080 con 0x7f072806fc80 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.601+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07280b4640 con 0x7f072806fc80 2024-04-12T19:50:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.602+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0728068c80 con 0x7f072806fc80 2024-04-12T19:50:59.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.603+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f071801a070 con 0x7f072806fc80 2024-04-12T19:50:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.604+0000 7f071ea00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:50:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.604+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f071808af30 con 0x7f072806fc80 2024-04-12T19:50:59.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.604+0000 7f0726000640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:50:59.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.605+0000 7f0726000640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0710036000 tx=0x7f071003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:50:59.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.609+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07180592c0 con 0x7f072806fc80 2024-04-12T19:50:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.813+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0728145a20 con 0x7f072806fc80 2024-04-12T19:50:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.814+0000 7f071ea00640 1 -- 172.21.15.32:0/3608027812 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0718058bc0 con 0x7f072806fc80 2024-04-12T19:50:59.813 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.816+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 msgr2=0x7f070406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.816+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 0x7f070406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0710036000 tx=0x7f071003b040 comp rx=0 tx=0).stop 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.816+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 msgr2=0x7f0728144ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f071800c400 tx=0x7f071800c8c0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 shutdown_connections 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f070406c8f0 0x7f070406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f072815a440 0x7f07281e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0728072150 0x7f07281e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 --2- 172.21.15.32:0/3608027812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f072806fc80 0x7f0728144ae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:50:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 >> 172.21.15.32:0/3608027812 conn(0x7f07280b2460 msgr2=0x7f072815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:50:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 shutdown_connections 2024-04-12T19:50:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:50:59.817+0000 7f072da00640 1 -- 172.21.15.32:0/3608027812 wait complete. 2024-04-12T19:50:59.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.063+0000 7fba23e00640 1 Processor -- start 2024-04-12T19:51:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.063+0000 7fba23e00640 1 -- start start 2024-04-12T19:51:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.064+0000 7fba23e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 0x7fba2409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.064+0000 7fba23e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba241603c0 0x7fba2409af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.064+0000 7fba23e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba240d03c0 con 0x7fba2409ba80 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.064+0000 7fba23e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba2415e1c0 con 0x7fba241603c0 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.064+0000 7fba23e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba2415d060 con 0x7fba2415c840 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba241603c0 0x7fba2409af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba241603c0 0x7fba2409af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53170/0 (socket says 172.21.15.32:53170) 2024-04-12T19:51:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 -- 172.21.15.32:0/3943965334 learned_addr learned my addr 172.21.15.32:0/3943965334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22a00640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 0x7fba2409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 -- 172.21.15.32:0/3943965334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 msgr2=0x7fba2409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 0x7fba2409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 -- 172.21.15.32:0/3943965334 >> v1:172.21.15.169:6789/0 conn(0x7fba2409ba80 legacy=0x7fba2415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22000640 1 -- 172.21.15.32:0/3943965334 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2415d340 con 0x7fba241603c0 2024-04-12T19:51:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.065+0000 7fba22a00640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 0x7fba2409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.066+0000 7fba22000640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba241603c0 0x7fba2409af80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a29b7df29274411 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba21600640 1 -- 172.21.15.32:0/3943965334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1002ca80 con 0x7fba241603c0 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba21600640 1 -- 172.21.15.32:0/3943965334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1002cc20 con 0x7fba241603c0 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba21600640 1 -- 172.21.15.32:0/3943965334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba100343f0 con 0x7fba241603c0 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 -- 172.21.15.32:0/3943965334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba241603c0 msgr2=0x7fba2409af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba241603c0 0x7fba2409af80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002bee0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 -- 172.21.15.32:0/3943965334 shutdown_connections 2024-04-12T19:51:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba241603c0 0x7fba2409af80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 --2- 172.21.15.32:0/3943965334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba2415c840 0x7fba2409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.067+0000 7fba23e00640 1 -- 172.21.15.32:0/3943965334 >> 172.21.15.32:0/3943965334 conn(0x7fba240d2550 msgr2=0x7fba240d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.068+0000 7fba23e00640 1 -- 172.21.15.32:0/3943965334 shutdown_connections 2024-04-12T19:51:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.068+0000 7fba23e00640 1 -- 172.21.15.32:0/3943965334 wait complete. 2024-04-12T19:51:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.069+0000 7fba23e00640 1 Processor -- start 2024-04-12T19:51:00.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.070+0000 7fba23e00640 1 -- start start 2024-04-12T19:51:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 0x7fba2415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba241603c0 0x7fba240cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba2415dbe0 con 0x7fba2409ba80 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba2415d900 con 0x7fba2415c840 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba2415cc20 con 0x7fba241603c0 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53174/0 (socket says 172.21.15.32:53174) 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 -- 172.21.15.32:0/1409384655 learned_addr learned my addr 172.21.15.32:0/1409384655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22000640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 0x7fba2415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba23400640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba241603c0 0x7fba240cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 -- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba241603c0 msgr2=0x7fba240cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba241603c0 0x7fba240cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 -- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 msgr2=0x7fba2415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 0x7fba2415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22a00640 1 -- 172.21.15.32:0/1409384655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2415b1d0 con 0x7fba2409ba80 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.071+0000 7fba22000640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 0x7fba2415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba22a00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c7f0 tx=0x7fba1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14015070 con 0x7fba2409ba80 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba14007a30 con 0x7fba2409ba80 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14007e00 con 0x7fba2409ba80 2024-04-12T19:51:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2415b4b0 con 0x7fba2409ba80 2024-04-12T19:51:00.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.072+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba240b5730 con 0x7fba2409ba80 2024-04-12T19:51:00.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.074+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba240755c0 con 0x7fba2409ba80 2024-04-12T19:51:00.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.074+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba14018070 con 0x7fba2409ba80 2024-04-12T19:51:00.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.075+0000 7fba1b400640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.075+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fba1408bb60 con 0x7fba2409ba80 2024-04-12T19:51:00.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.076+0000 7fba22000640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.076+0000 7fba22000640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba10035000 tx=0x7fba10030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.080+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba14059f90 con 0x7fba2409ba80 2024-04-12T19:51:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.286+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba240d32d0 con 0x7fba2409ba80 2024-04-12T19:51:00.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.286+0000 7fba1b400640 1 -- 172.21.15.32:0/1409384655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fba14059890 con 0x7fba2409ba80 2024-04-12T19:51:00.285 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:00.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 msgr2=0x7fba0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba10035000 tx=0x7fba10030000 comp rx=0 tx=0).stop 2024-04-12T19:51:00.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 msgr2=0x7fba24157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c7f0 tx=0x7fba1400cbf0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 shutdown_connections 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba241603c0 0x7fba240cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba2415c840 0x7fba2415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 --2- 172.21.15.32:0/1409384655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409ba80 0x7fba24157380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 >> 172.21.15.32:0/1409384655 conn(0x7fba240d2550 msgr2=0x7fba24070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 shutdown_connections 2024-04-12T19:51:00.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.289+0000 7fba23e00640 1 -- 172.21.15.32:0/1409384655 wait complete. 2024-04-12T19:51:00.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.523+0000 7f7c76600640 1 Processor -- start 2024-04-12T19:51:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.524+0000 7f7c76600640 1 -- start start 2024-04-12T19:51:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.525+0000 7f7c76600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c7015c680 0x7f7c70162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.525+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c7015f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.525+0000 7f7c76600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c700b70d0 con 0x7f7c7015fd90 2024-04-12T19:51:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.525+0000 7f7c76600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7c70162dd0 con 0x7f7c7015c680 2024-04-12T19:51:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.525+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c701630b0 con 0x7f7c700b6c70 2024-04-12T19:51:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.526+0000 7f7c75200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c7015c680 0x7f7c70162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.526+0000 7f7c75200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c7015c680 0x7f7c70162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:39986/0 (socket says 172.21.15.32:39986) 2024-04-12T19:51:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.526+0000 7f7c75200640 1 -- 172.21.15.32:0/4153728429 learned_addr learned my addr 172.21.15.32:0/4153728429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.526+0000 7f7c6fe00640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c7015f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.527+0000 7f7c75200640 1 -- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 msgr2=0x7f7c7015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.527+0000 7f7c75200640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c7015f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.527+0000 7f7c75200640 1 -- 172.21.15.32:0/4153728429 >> v1:172.21.15.169:6789/0 conn(0x7f7c7015fd90 legacy=0x7f7c700d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.527+0000 7f7c75200640 1 -- 172.21.15.32:0/4153728429 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c70163390 con 0x7f7c7015c680 2024-04-12T19:51:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.527+0000 7f7c6fe00640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c7015f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.528+0000 7f7c75200640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c70162730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009870 tx=0x7f7c5802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11a76f63c8c5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.529+0000 7f7c6f400640 1 -- 172.21.15.32:0/4153728429 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58034070 con 0x7f7c7015c680 2024-04-12T19:51:00.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.529+0000 7f7c6f400640 1 -- 172.21.15.32:0/4153728429 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c58030840 con 0x7f7c7015c680 2024-04-12T19:51:00.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.529+0000 7f7c6f400640 1 -- 172.21.15.32:0/4153728429 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58030c10 con 0x7f7c7015c680 2024-04-12T19:51:00.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 -- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 msgr2=0x7f7c70162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c70162730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009870 tx=0x7f7c5802b980 comp rx=0 tx=0).stop 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 -- 172.21.15.32:0/4153728429 shutdown_connections 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c7015f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 --2- 172.21.15.32:0/4153728429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c70162730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 -- 172.21.15.32:0/4153728429 >> 172.21.15.32:0/4153728429 conn(0x7f7c70073480 msgr2=0x7f7c70073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 -- 172.21.15.32:0/4153728429 shutdown_connections 2024-04-12T19:51:00.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.530+0000 7f7c76600640 1 -- 172.21.15.32:0/4153728429 wait complete. 2024-04-12T19:51:00.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.532+0000 7f7c76600640 1 Processor -- start 2024-04-12T19:51:00.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.532+0000 7f7c76600640 1 -- start start 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c701e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c700d3260 con 0x7f7c7015fd90 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7015ca60 con 0x7f7c7015c680 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c76600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c700d33e0 con 0x7f7c700b6c70 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c75c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c75c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53180/0 (socket says 172.21.15.32:53180) 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.533+0000 7f7c75c00640 1 -- 172.21.15.32:0/187284457 learned_addr learned my addr 172.21.15.32:0/187284457 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:00.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75200640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c6fe00640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c701e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 -- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 msgr2=0x7f7c700ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c700ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 -- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 msgr2=0x7f7c701e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c701e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 -- 172.21.15.32:0/187284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c700c8f20 con 0x7f7c7015fd90 2024-04-12T19:51:00.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75200640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c700ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c75c00640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c3c0 tx=0x7f7c6800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.534+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68017070 con 0x7f7c7015fd90 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.535+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c680052d0 con 0x7f7c7015fd90 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.535+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003830 con 0x7f7c7015fd90 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.535+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c700c9200 con 0x7f7c7015fd90 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.535+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c700c97c0 con 0x7f7c7015fd90 2024-04-12T19:51:00.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.536+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c700755c0 con 0x7f7c7015fd90 2024-04-12T19:51:00.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.537+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c6801a070 con 0x7f7c7015fd90 2024-04-12T19:51:00.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.538+0000 7f7c6cc00640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.538+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7c6808af70 con 0x7f7c7015fd90 2024-04-12T19:51:00.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.538+0000 7f7c75200640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.539+0000 7f7c75200640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c58036000 tx=0x7f7c5803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.542+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c68059300 con 0x7f7c7015fd90 2024-04-12T19:51:00.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.743+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c7006dbe0 con 0x7f7c7015fd90 2024-04-12T19:51:00.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.744+0000 7f7c6cc00640 1 -- 172.21.15.32:0/187284457 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f7c68058c00 con 0x7f7c7015fd90 2024-04-12T19:51:00.743 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 msgr2=0x7f7c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c58036000 tx=0x7f7c5803b040 comp rx=0 tx=0).stop 2024-04-12T19:51:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 msgr2=0x7f7c701e9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c3c0 tx=0x7f7c6800c7c0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 shutdown_connections 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7c4c06c8f0 0x7f7c4c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c7015fd90 0x7f7c701e9480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7015c680 0x7f7c701e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 --2- 172.21.15.32:0/187284457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c700b6c70 0x7f7c700ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 >> 172.21.15.32:0/187284457 conn(0x7f7c70073480 msgr2=0x7f7c70160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 shutdown_connections 2024-04-12T19:51:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.747+0000 7f7c76600640 1 -- 172.21.15.32:0/187284457 wait complete. 2024-04-12T19:51:00.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.972+0000 7f5677e00640 1 Processor -- start 2024-04-12T19:51:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.973+0000 7f5677e00640 1 -- start start 2024-04-12T19:51:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.974+0000 7f5677e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5678162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.974+0000 7f5677e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56780d0090 con 0x7f5678162020 2024-04-12T19:51:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.974+0000 7f5677e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56780cfad0 con 0x7f56780d1c90 2024-04-12T19:51:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.974+0000 7f5677e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5678163440 con 0x7f56780cf110 2024-04-12T19:51:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5676a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5678162020 0x7f56780d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37418/0 (socket says 172.21.15.32:37418) 2024-04-12T19:51:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5676a00640 1 -- 172.21.15.32:0/3297521851 learned_addr learned my addr 172.21.15.32:0/3297521851 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5677400640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5678162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201332260 0 0) 0x7f5668001110 con 0x7f5678162020 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5658003520 con 0x7f5678162020 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2873404791 0 0) 0x7f5658003520 con 0x7f5678162020 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 msgr2=0x7f5678162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.975+0000 7f5675600640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5678162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 >> v1:172.21.15.169:6789/0 conn(0x7f56780d1c90 legacy=0x7f56780d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780cfdb0 con 0x7f5678162020 2024-04-12T19:51:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f5668001840 con 0x7f5678162020 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 >> v1:172.21.15.32:6789/0 conn(0x7f5678162020 legacy=0x7f56780d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56580039e0 0x7f5658003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5658004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5658004d40 0x7f56580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5678163440 con 0x7f5658004d40 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f565800ba70 con 0x7f56580039e0 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f565800bc70 con 0x7f56780cf110 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5658004d40 0x7f56580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5676000640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5658004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 -- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 msgr2=0x7f5658004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5676a00640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56580039e0 0x7f5658003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56780cf110 0x7f5658004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 -- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56580039e0 msgr2=0x7f5658003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56580039e0 0x7f5658003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5677400640 1 -- 172.21.15.32:0/3297521851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5658003520 con 0x7f5658004d40 2024-04-12T19:51:00.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.976+0000 7f5676a00640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56580039e0 0x7f5658003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677400640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5658004d40 0x7f56580051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f56780cf560 tx=0x7f5658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505d0369d3a393ac server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039070 con 0x7f5658004d40 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567002cb90 con 0x7f5658004d40 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5675600640 1 -- 172.21.15.32:0/3297521851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670009c10 con 0x7f5658004d40 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677e00640 1 -- 172.21.15.32:0/3297521851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5658004d40 msgr2=0x7f56580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677e00640 1 --2- 172.21.15.32:0/3297521851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5658004d40 0x7f56580051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f56780cf560 tx=0x7f5658005830 comp rx=0 tx=0).stop 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677400640 1 -- 172.21.15.32:0/3297521851 reap_dead start 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677e00640 1 -- 172.21.15.32:0/3297521851 shutdown_connections 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.978+0000 7f5677e00640 1 -- 172.21.15.32:0/3297521851 >> 172.21.15.32:0/3297521851 conn(0x7f567806cf00 msgr2=0x7f567806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.979+0000 7f5677e00640 1 -- 172.21.15.32:0/3297521851 shutdown_connections 2024-04-12T19:51:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.979+0000 7f5677e00640 1 -- 172.21.15.32:0/3297521851 wait complete. 2024-04-12T19:51:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.980+0000 7f5677e00640 1 Processor -- start 2024-04-12T19:51:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.981+0000 7f5677e00640 1 -- start start 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5658004d40 0x7f56780ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 0x7f56780cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56780cf880 con 0x7f56780cb8f0 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b7b30 con 0x7f56780cd940 2024-04-12T19:51:00.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56780d2e70 con 0x7f5658004d40 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53200/0 (socket says 172.21.15.32:53200) 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 -- 172.21.15.32:0/2392219751 learned_addr learned my addr 172.21.15.32:0/2392219751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676a00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5658004d40 0x7f56780ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677400640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 0x7f56780cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 -- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5658004d40 msgr2=0x7f56780ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5658004d40 0x7f56780ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 -- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 msgr2=0x7f56780cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 0x7f56780cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5676000640 1 -- 172.21.15.32:0/2392219751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780cbd70 con 0x7f56780cb8f0 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.982+0000 7f5677400640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 0x7f56780cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f5676000640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f566000d570 tx=0x7f566000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660005480 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5660017070 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660003810 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56780cc040 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.983+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56780cc600 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.984+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678160490 con 0x7f56780cb8f0 2024-04-12T19:51:00.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.985+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56600039b0 con 0x7f56780cb8f0 2024-04-12T19:51:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.986+0000 7f566ea00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.986+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f566008b9d0 con 0x7f56780cb8f0 2024-04-12T19:51:00.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.986+0000 7f5676a00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:00.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.987+0000 7f5676a00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 0x7f565406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56680012b0 tx=0x7f5668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:00.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:00.991+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5678160490 con 0x7f56780cb8f0 2024-04-12T19:51:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.199+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56780755c0 con 0x7f56780cb8f0 2024-04-12T19:51:01.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.202+0000 7f566ea00640 1 -- 172.21.15.32:0/2392219751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7f5660059df0 con 0x7f56780cb8f0 2024-04-12T19:51:01.202 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:01.202 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.205+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 msgr2=0x7f565406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.205+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 0x7f565406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56680012b0 tx=0x7f5668009040 comp rx=0 tx=0).stop 2024-04-12T19:51:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.205+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 msgr2=0x7f567814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.205+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f566000d570 tx=0x7f566000da30 comp rx=0 tx=0).stop 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 shutdown_connections 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56780cd940 0x7f56780cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56780cb8f0 0x7f567814ef10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 --2- 172.21.15.32:0/2392219751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5658004d40 0x7f56780ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 >> 172.21.15.32:0/2392219751 conn(0x7f567806cf00 msgr2=0x7f567815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 shutdown_connections 2024-04-12T19:51:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.206+0000 7f5677e00640 1 -- 172.21.15.32:0/2392219751 wait complete. 2024-04-12T19:51:01.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:01.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.458+0000 7f5d47200640 1 Processor -- start 2024-04-12T19:51:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.458+0000 7f5d47200640 1 -- start start 2024-04-12T19:51:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d47200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d40162f90 0x7f5d400ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d47200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 0x7f5d40166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d47200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d400cec10 con 0x7f5d400afbc0 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d47200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5d400b46c0 con 0x7f5d40162f90 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d47200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4006fe90 con 0x7f5d40072860 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.460+0000 7f5d3fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d40162f90 0x7f5d400ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d40162f90 0x7f5d400ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53214/0 (socket says 172.21.15.32:53214) 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3998945533 learned_addr learned my addr 172.21.15.32:0/3998945533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3f400640 1 --1- 172.21.15.32:0/3998945533 >> v1:172.21.15.32:6789/0 conn(0x7f5d400afbc0 0x7f5d40072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37428/0 (socket says 172.21.15.32:37428) 2024-04-12T19:51:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3998945533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 msgr2=0x7f5d40166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d45200640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 0x7f5d40166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 0x7f5d40166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3998945533 >> v1:172.21.15.32:6789/0 conn(0x7f5d400afbc0 legacy=0x7f5d40072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3998945533 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d400b3ab0 con 0x7f5d40162f90 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d45200640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 0x7f5d40166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.461+0000 7f5d3ea00640 1 -- 172.21.15.32:0/3998945533 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405032471 0 0) 0x7f5d400cec10 con 0x7f5d400afbc0 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.462+0000 7f5d3fe00640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40162f90 0x7f5d400ce6d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009870 tx=0x7f5d3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52043673dd167442 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.462+0000 7f5d3ea00640 1 -- 172.21.15.32:0/3998945533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38034070 con 0x7f5d40162f90 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d3ea00640 1 -- 172.21.15.32:0/3998945533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38030840 con 0x7f5d40162f90 2024-04-12T19:51:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d3ea00640 1 -- 172.21.15.32:0/3998945533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38030c10 con 0x7f5d40162f90 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 -- 172.21.15.32:0/3998945533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40162f90 msgr2=0x7f5d400ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40162f90 0x7f5d400ce6d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009870 tx=0x7f5d3802b980 comp rx=0 tx=0).stop 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 -- 172.21.15.32:0/3998945533 shutdown_connections 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d40072860 0x7f5d40166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 --2- 172.21.15.32:0/3998945533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40162f90 0x7f5d400ce6d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 -- 172.21.15.32:0/3998945533 >> 172.21.15.32:0/3998945533 conn(0x7f5d400b2460 msgr2=0x7f5d400b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 -- 172.21.15.32:0/3998945533 shutdown_connections 2024-04-12T19:51:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.463+0000 7f5d47200640 1 -- 172.21.15.32:0/3998945533 wait complete. 2024-04-12T19:51:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.465+0000 7f5d47200640 1 Processor -- start 2024-04-12T19:51:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.465+0000 7f5d47200640 1 -- start start 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d400afbc0 0x7f5d4015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d40162f90 0x7f5d4006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d400b4350 con 0x7f5d40072860 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d400b4070 con 0x7f5d40162f90 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d47200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d40163370 con 0x7f5d400afbc0 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d3fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d3fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53228/0 (socket says 172.21.15.32:53228) 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.466+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3465215314 learned_addr learned my addr 172.21.15.32:0/3465215314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:01.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d400afbc0 msgr2=0x7f5d4015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d400afbc0 0x7f5d4015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d40162f90 msgr2=0x7f5d4006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d40162f90 0x7f5d4006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 -- 172.21.15.32:0/3465215314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d400a9db0 con 0x7f5d40072860 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d3fe00640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009870 tx=0x7f5d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.467+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38034040 con 0x7f5d40072860 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.468+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d38039d90 con 0x7f5d40072860 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.468+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38043610 con 0x7f5d40072860 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.468+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d400aa070 con 0x7f5d40072860 2024-04-12T19:51:01.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.468+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d400aa910 con 0x7f5d40072860 2024-04-12T19:51:01.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.469+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d40069540 con 0x7f5d40072860 2024-04-12T19:51:01.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.470+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d3804a060 con 0x7f5d40072860 2024-04-12T19:51:01.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.471+0000 7f5d37e00640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.471+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5d380add00 con 0x7f5d40072860 2024-04-12T19:51:01.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.471+0000 7f5d3f400640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.472+0000 7f5d3f400640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007fd0 tx=0x7f5d2c007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.476+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d40069540 con 0x7f5d40072860 2024-04-12T19:51:01.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.683+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d40068c80 con 0x7f5d40072860 2024-04-12T19:51:01.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.684+0000 7f5d37e00640 1 -- 172.21.15.32:0/3465215314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f5d3803e070 con 0x7f5d40072860 2024-04-12T19:51:01.684 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 msgr2=0x7f5d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 0x7f5d2006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c007fd0 tx=0x7f5d2c007920 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 msgr2=0x7f5d401f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009870 tx=0x7f5d380046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 shutdown_connections 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5d2006c8f0 0x7f5d2006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d40162f90 0x7f5d4006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d400afbc0 0x7f5d4015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 --2- 172.21.15.32:0/3465215314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d40072860 0x7f5d401f8fd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 >> 172.21.15.32:0/3465215314 conn(0x7f5d400b2460 msgr2=0x7f5d400b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 shutdown_connections 2024-04-12T19:51:01.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.687+0000 7f5d47200640 1 -- 172.21.15.32:0/3465215314 wait complete. 2024-04-12T19:51:01.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.938+0000 7f6c1d800640 1 Processor -- start 2024-04-12T19:51:01.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.939+0000 7f6c1d800640 1 -- start start 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c1815c680 0x7f6c18162730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c180b6c70 0x7f6c1815f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c180d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6c180b70d0 con 0x7f6c180b6c70 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6c18162dd0 con 0x7f6c1815c680 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.940+0000 7f6c1d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c181630b0 con 0x7f6c1815fd90 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c180b6c70 0x7f6c1815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c180b6c70 0x7f6c1815f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53244/0 (socket says 172.21.15.32:53244) 2024-04-12T19:51:01.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 -- 172.21.15.32:0/3130514452 learned_addr learned my addr 172.21.15.32:0/3130514452 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 -- 172.21.15.32:0/3130514452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 msgr2=0x7f6c180d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 --2- 172.21.15.32:0/3130514452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c180d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 -- 172.21.15.32:0/3130514452 >> v2:172.21.15.169:3300/0 conn(0x7f6c1815c680 msgr2=0x7f6c18162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 --2- 172.21.15.32:0/3130514452 >> v2:172.21.15.169:3300/0 conn(0x7f6c1815c680 0x7f6c18162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.941+0000 7f6c17400640 1 -- 172.21.15.32:0/3130514452 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18163390 con 0x7f6c180b6c70 2024-04-12T19:51:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.942+0000 7f6c17400640 1 --2- 172.21.15.32:0/3130514452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c1815f850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c180b73a0 tx=0x7f6c0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7703131efd6442 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c16a00640 1 -- 172.21.15.32:0/3130514452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00035020 con 0x7f6c180b6c70 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c16a00640 1 -- 172.21.15.32:0/3130514452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c00031870 con 0x7f6c180b6c70 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c16a00640 1 -- 172.21.15.32:0/3130514452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00031c40 con 0x7f6c180b6c70 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c1d800640 1 -- 172.21.15.32:0/3130514452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 msgr2=0x7f6c1815f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3130514452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c1815f850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c180b73a0 tx=0x7f6c0002b910 comp rx=0 tx=0).stop 2024-04-12T19:51:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.943+0000 7f6c1d800640 1 -- 172.21.15.32:0/3130514452 shutdown_connections 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3130514452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c1815f850 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3130514452 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c180d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3130514452 >> v2:172.21.15.169:3300/0 conn(0x7f6c1815c680 0x7f6c18162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 -- 172.21.15.32:0/3130514452 >> 172.21.15.32:0/3130514452 conn(0x7f6c18073480 msgr2=0x7f6c18073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 -- 172.21.15.32:0/3130514452 shutdown_connections 2024-04-12T19:51:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.944+0000 7f6c1d800640 1 -- 172.21.15.32:0/3130514452 wait complete. 2024-04-12T19:51:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.945+0000 7f6c1d800640 1 Processor -- start 2024-04-12T19:51:01.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.946+0000 7f6c1d800640 1 -- start start 2024-04-12T19:51:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c1815c680 0x7f6c180ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c181e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180d3090 con 0x7f6c180b6c70 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1815ca60 con 0x7f6c1815c680 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c180d3210 con 0x7f6c1815fd90 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53248/0 (socket says 172.21.15.32:53248) 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 -- 172.21.15.32:0/3973769511 learned_addr learned my addr 172.21.15.32:0/3973769511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17400640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c1815c680 0x7f6c180ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c1ce00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c181e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 -- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 msgr2=0x7f6c181e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c181e95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 -- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c1815c680 msgr2=0x7f6c180ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.947+0000 7f6c17e00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c1815c680 0x7f6c180ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c17e00640 1 -- 172.21.15.32:0/3973769511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18155c60 con 0x7f6c180b6c70 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c1ce00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c181e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c17e00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ba50 tx=0x7f6c0800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08019070 con 0x7f6c180b6c70 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0800cc00 con 0x7f6c180b6c70 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08014440 con 0x7f6c180b6c70 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c180c9140 con 0x7f6c180b6c70 2024-04-12T19:51:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.948+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c180c9700 con 0x7f6c180b6c70 2024-04-12T19:51:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.950+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c180755c0 con 0x7f6c180b6c70 2024-04-12T19:51:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.951+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c0801c070 con 0x7f6c180b6c70 2024-04-12T19:51:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.952+0000 7f6c0fe00640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 0x7f6bf406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.952+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c0808b890 con 0x7f6c180b6c70 2024-04-12T19:51:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.952+0000 7f6c17400640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 0x7f6bf406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.953+0000 7f6c17400640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 0x7f6bf406ee30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c0003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:01.956+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c08059d10 con 0x7f6c180b6c70 2024-04-12T19:51:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.168+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c180b3e30 con 0x7f6c180b6c70 2024-04-12T19:51:02.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.169+0000 7f6c0fe00640 1 -- 172.21.15.32:0/3973769511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f6c08059610 con 0x7f6c180b6c70 2024-04-12T19:51:02.169 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.171+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 msgr2=0x7f6bf406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.171+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 0x7f6bf406ee30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c0003c040 comp rx=0 tx=0).stop 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 msgr2=0x7f6c18155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ba50 tx=0x7f6c0800bf10 comp rx=0 tx=0).stop 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 shutdown_connections 2024-04-12T19:51:02.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6bf406c970 0x7f6bf406ee30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c1815fd90 0x7f6c181e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c1815c680 0x7f6c180ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 --2- 172.21.15.32:0/3973769511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c180b6c70 0x7f6c18155720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 >> 172.21.15.32:0/3973769511 conn(0x7f6c18073480 msgr2=0x7f6c180d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 shutdown_connections 2024-04-12T19:51:02.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.172+0000 7f6c1d800640 1 -- 172.21.15.32:0/3973769511 wait complete. 2024-04-12T19:51:02.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:02.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.426+0000 7f5e6be00640 1 Processor -- start 2024-04-12T19:51:02.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.427+0000 7f5e6be00640 1 -- start start 2024-04-12T19:51:02.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.428+0000 7f5e6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5e6415ecc0 0x7f5e640d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.428+0000 7f5e6be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640d3270 0x7f5e640b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.428+0000 7f5e6be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e640ba4d0 con 0x7f5e640b3720 2024-04-12T19:51:02.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.428+0000 7f5e6be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5e6415f0a0 con 0x7f5e640d3270 2024-04-12T19:51:02.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.428+0000 7f5e6be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5e640ba280 con 0x7f5e6415ecc0 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.429+0000 7f5e68a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640d3270 0x7f5e640b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.429+0000 7f5e68a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e640d3270 0x7f5e640b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.429+0000 7f5e68a00640 1 -- 172.21.15.32:0/1401034055 learned_addr learned my addr 172.21.15.32:0/1401034055 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.429+0000 7f5e69400640 1 --2- 172.21.15.32:0/1401034055 >> v2:172.21.15.169:3300/0 conn(0x7f5e6415ecc0 0x7f5e640d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.430+0000 7f5e68a00640 1 -- 172.21.15.32:0/1401034055 >> v2:172.21.15.169:3300/0 conn(0x7f5e6415ecc0 msgr2=0x7f5e640d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.430+0000 7f5e68a00640 1 --2- 172.21.15.32:0/1401034055 >> v2:172.21.15.169:3300/0 conn(0x7f5e6415ecc0 0x7f5e640d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.430+0000 7f5e68a00640 1 -- 172.21.15.32:0/1401034055 >> v1:172.21.15.32:6789/0 conn(0x7f5e640b3720 legacy=0x7f5e640cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.430+0000 7f5e68a00640 1 -- 172.21.15.32:0/1401034055 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640ba7b0 con 0x7f5e640d3270 2024-04-12T19:51:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.430+0000 7f5e63e00640 1 -- 172.21.15.32:0/1401034055 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516300828 0 0) 0x7f5e640ba4d0 con 0x7f5e640b3720 2024-04-12T19:51:02.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.431+0000 7f5e68a00640 1 --2- 172.21.15.32:0/1401034055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640d3270 0x7f5e640b95f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e640cf1b0 tx=0x7f5e5002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51b118693cd9aed server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.431+0000 7f5e63e00640 1 -- 172.21.15.32:0/1401034055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50034070 con 0x7f5e640d3270 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.431+0000 7f5e63e00640 1 -- 172.21.15.32:0/1401034055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e50030860 con 0x7f5e640d3270 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.431+0000 7f5e63e00640 1 -- 172.21.15.32:0/1401034055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50030c30 con 0x7f5e640d3270 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 -- 172.21.15.32:0/1401034055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640d3270 msgr2=0x7f5e640b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 --2- 172.21.15.32:0/1401034055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640d3270 0x7f5e640b95f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e640cf1b0 tx=0x7f5e5002b9a0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 -- 172.21.15.32:0/1401034055 shutdown_connections 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 --2- 172.21.15.32:0/1401034055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640d3270 0x7f5e640b95f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 --2- 172.21.15.32:0/1401034055 >> v2:172.21.15.169:3300/0 conn(0x7f5e6415ecc0 0x7f5e640d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 -- 172.21.15.32:0/1401034055 >> 172.21.15.32:0/1401034055 conn(0x7f5e64073480 msgr2=0x7f5e64073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 -- 172.21.15.32:0/1401034055 shutdown_connections 2024-04-12T19:51:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.432+0000 7f5e6be00640 1 -- 172.21.15.32:0/1401034055 wait complete. 2024-04-12T19:51:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.434+0000 7f5e6be00640 1 Processor -- start 2024-04-12T19:51:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.434+0000 7f5e6be00640 1 -- start start 2024-04-12T19:51:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e640d3270 0x7f5e640ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e6415ecc0 0x7f5e640ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640b8aa0 con 0x7f5e640b3720 2024-04-12T19:51:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640b90d0 con 0x7f5e640d3270 2024-04-12T19:51:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e6be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e640cfdb0 con 0x7f5e6415ecc0 2024-04-12T19:51:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e69400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e69400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53268/0 (socket says 172.21.15.32:53268) 2024-04-12T19:51:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.435+0000 7f5e69400640 1 -- 172.21.15.32:0/727745633 learned_addr learned my addr 172.21.15.32:0/727745633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 -- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e6415ecc0 msgr2=0x7f5e640ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69e00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e6415ecc0 0x7f5e640ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e68a00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e640d3270 0x7f5e640ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e6415ecc0 0x7f5e640ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 -- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e640d3270 msgr2=0x7f5e640ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e640d3270 0x7f5e640ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 -- 172.21.15.32:0/727745633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640b59d0 con 0x7f5e640b3720 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e69400640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400e8a0 tx=0x7f5e5400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54017070 con 0x7f5e640b3720 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5400c940 con 0x7f5e640b3720 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6415f820 con 0x7f5e640b3720 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.436+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400cd30 con 0x7f5e640b3720 2024-04-12T19:51:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.437+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6415fde0 con 0x7f5e640b3720 2024-04-12T19:51:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.438+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e64068c80 con 0x7f5e640b3720 2024-04-12T19:51:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.439+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e5401a070 con 0x7f5e640b3720 2024-04-12T19:51:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.439+0000 7f5e61600640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.440+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5e5408ba40 con 0x7f5e640b3720 2024-04-12T19:51:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.440+0000 7f5e68a00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.441+0000 7f5e68a00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 0x7f5e4406ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e50036000 tx=0x7f5e5003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.444+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e640b59d0 con 0x7f5e640b3720 2024-04-12T19:51:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.647+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e6415f4d0 con 0x7f5e640b3720 2024-04-12T19:51:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.648+0000 7f5e61600640 1 -- 172.21.15.32:0/727745633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f5e54059dd0 con 0x7f5e640b3720 2024-04-12T19:51:02.648 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 msgr2=0x7f5e4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 0x7f5e4406ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e50036000 tx=0x7f5e5003b040 comp rx=0 tx=0).stop 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 msgr2=0x7f5e640d3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400e8a0 tx=0x7f5e5400ed60 comp rx=0 tx=0).stop 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 shutdown_connections 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e6415ecc0 0x7f5e640ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e640d3270 0x7f5e640ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.650+0000 7f5e6be00640 1 --2- 172.21.15.32:0/727745633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e640b3720 0x7f5e640d3d60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.651+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 >> 172.21.15.32:0/727745633 conn(0x7f5e64073480 msgr2=0x7f5e640b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.651+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 shutdown_connections 2024-04-12T19:51:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.651+0000 7f5e6be00640 1 -- 172.21.15.32:0/727745633 wait complete. 2024-04-12T19:51:02.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:02.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.898+0000 7fedb6600640 1 Processor -- start 2024-04-12T19:51:02.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.898+0000 7fedb6600640 1 -- start start 2024-04-12T19:51:02.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb00cee00 0x7fedb006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb0071f50 0x7fedb00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fedb015cbe0 0x7fedb015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fedb006f4f0 con 0x7fedb00cee00 2024-04-12T19:51:02.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fedb00728f0 con 0x7fedb015cbe0 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fedb015dbb0 con 0x7fedb0071f50 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb00cee00 0x7fedb006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb00cee00 0x7fedb006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53278/0 (socket says 172.21.15.32:53278) 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.900+0000 7fedb5200640 1 -- 172.21.15.32:0/2989460409 learned_addr learned my addr 172.21.15.32:0/2989460409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5c00640 1 --2- 172.21.15.32:0/2989460409 >> v2:172.21.15.169:3300/0 conn(0x7fedb015cbe0 0x7fedb015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedafe00640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb0071f50 0x7fedb00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5200640 1 -- 172.21.15.32:0/2989460409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb0071f50 msgr2=0x7fedb00723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5200640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb0071f50 0x7fedb00723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5200640 1 -- 172.21.15.32:0/2989460409 >> v2:172.21.15.169:3300/0 conn(0x7fedb015cbe0 msgr2=0x7fedb015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5200640 1 --2- 172.21.15.32:0/2989460409 >> v2:172.21.15.169:3300/0 conn(0x7fedb015cbe0 0x7fedb015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.901+0000 7fedb5200640 1 -- 172.21.15.32:0/2989460409 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb015d8d0 con 0x7fedb00cee00 2024-04-12T19:51:02.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.903+0000 7fedb5200640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00cee00 0x7fedb006e8c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feda4009870 tx=0x7feda402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187e5a2a3caa2a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.903+0000 7fedaf400640 1 -- 172.21.15.32:0/2989460409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035020 con 0x7fedb00cee00 2024-04-12T19:51:02.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.903+0000 7fedaf400640 1 -- 172.21.15.32:0/2989460409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4031640 con 0x7fedb00cee00 2024-04-12T19:51:02.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.903+0000 7fedaf400640 1 -- 172.21.15.32:0/2989460409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda402d040 con 0x7fedb00cee00 2024-04-12T19:51:02.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 -- 172.21.15.32:0/2989460409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00cee00 msgr2=0x7fedb006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00cee00 0x7fedb006e8c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feda4009870 tx=0x7feda402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 -- 172.21.15.32:0/2989460409 shutdown_connections 2024-04-12T19:51:02.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 --2- 172.21.15.32:0/2989460409 >> v2:172.21.15.169:3300/0 conn(0x7fedb015cbe0 0x7fedb015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb0071f50 0x7fedb00723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 --2- 172.21.15.32:0/2989460409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00cee00 0x7fedb006e8c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 -- 172.21.15.32:0/2989460409 >> 172.21.15.32:0/2989460409 conn(0x7fedb006b930 msgr2=0x7fedb006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 -- 172.21.15.32:0/2989460409 shutdown_connections 2024-04-12T19:51:02.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.904+0000 7fedb6600640 1 -- 172.21.15.32:0/2989460409 wait complete. 2024-04-12T19:51:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.906+0000 7fedb6600640 1 Processor -- start 2024-04-12T19:51:02.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.906+0000 7fedb6600640 1 -- start start 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedb00cee00 0x7fedb0149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb015cbe0 0x7fedb0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fedb006e3a0 con 0x7fedb0071f50 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedb006e0c0 con 0x7fedb00cee00 2024-04-12T19:51:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fedb015d5b0 con 0x7fedb015cbe0 2024-04-12T19:51:02.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53288/0 (socket says 172.21.15.32:53288) 2024-04-12T19:51:02.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.907+0000 7fedb5200640 1 -- 172.21.15.32:0/3193089652 learned_addr learned my addr 172.21.15.32:0/3193089652 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedafe00640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedb00cee00 0x7fedb0149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 -- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb015cbe0 msgr2=0x7fedb0152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb015cbe0 0x7fedb0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 -- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedb00cee00 msgr2=0x7fedb0149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedb00cee00 0x7fedb0149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 -- 172.21.15.32:0/3193089652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb015e110 con 0x7fedb0071f50 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb5200640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035040 con 0x7fedb0071f50 2024-04-12T19:51:02.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4039c10 con 0x7fedb0071f50 2024-04-12T19:51:02.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4045480 con 0x7fedb0071f50 2024-04-12T19:51:02.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb015e3b0 con 0x7fedb0071f50 2024-04-12T19:51:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.908+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb015e970 con 0x7fedb0071f50 2024-04-12T19:51:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.910+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb00755c0 con 0x7fedb0071f50 2024-04-12T19:51:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.911+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feda4038020 con 0x7fedb0071f50 2024-04-12T19:51:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.911+0000 7fedacc00640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 0x7fed9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.912+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feda40ad870 con 0x7fedb0071f50 2024-04-12T19:51:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.912+0000 7fedafe00640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 0x7fed9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.913+0000 7fedafe00640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 0x7fed9406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feda0006fd0 tx=0x7feda0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:02.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:02.916+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feda4042330 con 0x7fedb0071f50 2024-04-12T19:51:03.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.120+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedb00af500 con 0x7fedb0071f50 2024-04-12T19:51:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.123+0000 7fedacc00640 1 -- 172.21.15.32:0/3193089652 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16597 (secure 0 0 0) 0x7feda407bd30 con 0x7fedb0071f50 2024-04-12T19:51:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:03.124 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:50:25.659253+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 msgr2=0x7fed9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 0x7fed9406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feda0006fd0 tx=0x7feda0008040 comp rx=0 tx=0).stop 2024-04-12T19:51:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 msgr2=0x7fedb0148460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feda4000c00 tx=0x7feda40046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 shutdown_connections 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fed9406c920 0x7fed9406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb015cbe0 0x7fedb0152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedb00cee00 0x7fedb0149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 --2- 172.21.15.32:0/3193089652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb0071f50 0x7fedb0148460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 >> 172.21.15.32:0/3193089652 conn(0x7fedb006b930 msgr2=0x7fedb006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 shutdown_connections 2024-04-12T19:51:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.126+0000 7fedb6600640 1 -- 172.21.15.32:0/3193089652 wait complete. 2024-04-12T19:51:03.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:03.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:03.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.371+0000 7f1f73000640 1 Processor -- start 2024-04-12T19:51:03.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.371+0000 7f1f73000640 1 -- start start 2024-04-12T19:51:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.372+0000 7f1f73000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f6c15d510 0x7f1f6c06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f73000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f73000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6c0b6300 con 0x7f1f6c15d8f0 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f73000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0cea20 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f73000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0cecd0 con 0x7f1f6c15d510 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f72600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f72600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53290/0 (socket says 172.21.15.32:53290) 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f72600640 1 -- 172.21.15.32:0/3490811641 learned_addr learned my addr 172.21.15.32:0/3490811641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.373+0000 7f1f71c00640 1 --2- 172.21.15.32:0/3490811641 >> v2:172.21.15.169:3300/0 conn(0x7f1f6c15d510 0x7f1f6c06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.374+0000 7f1f72600640 1 -- 172.21.15.32:0/3490811641 >> v2:172.21.15.169:3300/0 conn(0x7f1f6c15d510 msgr2=0x7f1f6c06ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.374+0000 7f1f72600640 1 --2- 172.21.15.32:0/3490811641 >> v2:172.21.15.169:3300/0 conn(0x7f1f6c15d510 0x7f1f6c06ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.374+0000 7f1f72600640 1 -- 172.21.15.32:0/3490811641 >> v1:172.21.15.169:6789/0 conn(0x7f1f6c15d8f0 legacy=0x7f1f6c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.374+0000 7f1f72600640 1 -- 172.21.15.32:0/3490811641 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c0cefb0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.375+0000 7f1f72600640 1 --2- 172.21.15.32:0/3490811641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b3460 tx=0x7f1f6402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf3d1d2a2dd1320 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.375+0000 7f1f6be00640 1 -- 172.21.15.32:0/3490811641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6402ca80 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.375+0000 7f1f6be00640 1 -- 172.21.15.32:0/3490811641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f6402cc20 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.375+0000 7f1f6be00640 1 -- 172.21.15.32:0/3490811641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640343f0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 -- 172.21.15.32:0/3490811641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 msgr2=0x7f1f6c0b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 --2- 172.21.15.32:0/3490811641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b3460 tx=0x7f1f6402bee0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 -- 172.21.15.32:0/3490811641 shutdown_connections 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 --2- 172.21.15.32:0/3490811641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c0b5dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 --2- 172.21.15.32:0/3490811641 >> v2:172.21.15.169:3300/0 conn(0x7f1f6c15d510 0x7f1f6c06ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 -- 172.21.15.32:0/3490811641 >> 172.21.15.32:0/3490811641 conn(0x7f1f6c0b23a0 msgr2=0x7f1f6c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 -- 172.21.15.32:0/3490811641 shutdown_connections 2024-04-12T19:51:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.376+0000 7f1f73000640 1 -- 172.21.15.32:0/3490811641 wait complete. 2024-04-12T19:51:03.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.378+0000 7f1f73000640 1 Processor -- start 2024-04-12T19:51:03.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.378+0000 7f1f73000640 1 -- start start 2024-04-12T19:51:03.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 0x7f1f6c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f6c150f40 0x7f1f6c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c06e560 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c06e280 con 0x7f1f6c15d510 2024-04-12T19:51:03.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f73000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c06efc0 con 0x7f1f6c150f40 2024-04-12T19:51:03.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f71c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f71c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53296/0 (socket says 172.21.15.32:53296) 2024-04-12T19:51:03.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.379+0000 7f1f71c00640 1 -- 172.21.15.32:0/2595807407 learned_addr learned my addr 172.21.15.32:0/2595807407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:03.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71200640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 0x7f1f6c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 -- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f6c150f40 msgr2=0x7f1f6c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f6c150f40 0x7f1f6c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 -- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 msgr2=0x7f1f6c0b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 0x7f1f6c0b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 -- 172.21.15.32:0/2595807407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c154ae0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71200640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 0x7f1f6c0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f71c00640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00d5f0 tx=0x7f1f5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0125f0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c019070 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0129d0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.380+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c154dc0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.381+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c1e6b60 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.382+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c0755c0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.383+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f5c012b70 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.383+0000 7f1f69600640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 0x7f1f4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.384+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1f5c08bf10 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.384+0000 7f1f71200640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 0x7f1f4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.384+0000 7f1f71200640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 0x7f1f4c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b2060 tx=0x7f1f58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.388+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f5c05a3d0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.597+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f6c0b5790 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.598+0000 7f1f69600640 1 -- 172.21.15.32:0/2595807407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1f5c059cd0 con 0x7f1f6c0b59b0 2024-04-12T19:51:03.599 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:03.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 msgr2=0x7f1f4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 0x7f1f4c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b2060 tx=0x7f1f58009040 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 msgr2=0x7f1f6c1e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00d5f0 tx=0x7f1f5c00dab0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 shutdown_connections 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1f4c06c8f0 0x7f1f4c06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f6c150f40 0x7f1f6c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f6c15d510 0x7f1f6c0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 --2- 172.21.15.32:0/2595807407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c0b59b0 0x7f1f6c1e6620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 >> 172.21.15.32:0/2595807407 conn(0x7f1f6c0b23a0 msgr2=0x7f1f6c0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 shutdown_connections 2024-04-12T19:51:03.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.601+0000 7f1f73000640 1 -- 172.21.15.32:0/2595807407 wait complete. 2024-04-12T19:51:03.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.836+0000 7f5605c00640 1 Processor -- start 2024-04-12T19:51:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.837+0000 7f5605c00640 1 -- start start 2024-04-12T19:51:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.838+0000 7f5605c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 0x7f5600072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.838+0000 7f5605c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5600072690 0x7f560015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.838+0000 7f5605c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560015ffe0 con 0x7f5600163520 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.838+0000 7f5605c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f56000afde0 con 0x7f5600072690 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.838+0000 7f5605c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56001627f0 con 0x7f5600145070 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55fe000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5600163520 0x7f560015f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37458/0 (socket says 172.21.15.32:37458) 2024-04-12T19:51:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55fe000640 1 -- 172.21.15.32:0/764134110 learned_addr learned my addr 172.21.15.32:0/764134110 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55fea00640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 0x7f5600072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55ff400640 1 --2- 172.21.15.32:0/764134110 >> v2:172.21.15.169:3300/0 conn(0x7f5600072690 0x7f560015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741671039 0 0) 0x7f560015ffe0 con 0x7f5600163520 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.839+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e4003520 con 0x7f5600163520 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3009805057 0 0) 0x7f55e4003520 con 0x7f5600163520 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 msgr2=0x7f5600072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 0x7f5600072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 >> v2:172.21.15.169:3300/0 conn(0x7f5600072690 msgr2=0x7f560015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 --2- 172.21.15.32:0/764134110 >> v2:172.21.15.169:3300/0 conn(0x7f5600072690 0x7f560015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5600162a40 con 0x7f5600163520 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f55ec001670 con 0x7f5600163520 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 >> v1:172.21.15.32:6789/0 conn(0x7f5600163520 legacy=0x7f560015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55e40039b0 0x7f55e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e4004800 0x7f55e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 0x7f55e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56000afde0 con 0x7f55e4004800 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56001627f0 con 0x7f55e40039b0 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55e400ba10 con 0x7f5600145070 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fe000640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e4004800 0x7f55e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fea00640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55e40039b0 0x7f55e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fe000640 1 -- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 msgr2=0x7f55e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fe000640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5600145070 0x7f55e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.840+0000 7f55fe000640 1 -- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55e40039b0 msgr2=0x7f55e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.841+0000 7f55fe000640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55e40039b0 0x7f55e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.841+0000 7f55fe000640 1 -- 172.21.15.32:0/764134110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e4003520 con 0x7f55e4004800 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f55fe000640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e4004800 0x7f55e4004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c1c0 tx=0x7f55e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f8b311918707fb server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec007c60 con 0x7f55e4004800 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ec030070 con 0x7f55e4004800 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f55fd600640 1 -- 172.21.15.32:0/764134110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec005d30 con 0x7f55e4004800 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f5605c00640 1 -- 172.21.15.32:0/764134110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e4004800 msgr2=0x7f55e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f5605c00640 1 --2- 172.21.15.32:0/764134110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e4004800 0x7f55e4004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c1c0 tx=0x7f55e4005830 comp rx=0 tx=0).stop 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.842+0000 7f55ff400640 1 -- 172.21.15.32:0/764134110 reap_dead start 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.843+0000 7f5605c00640 1 -- 172.21.15.32:0/764134110 shutdown_connections 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.843+0000 7f5605c00640 1 -- 172.21.15.32:0/764134110 >> 172.21.15.32:0/764134110 conn(0x7f56000b2460 msgr2=0x7f56000b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.843+0000 7f5605c00640 1 -- 172.21.15.32:0/764134110 shutdown_connections 2024-04-12T19:51:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.843+0000 7f5605c00640 1 -- 172.21.15.32:0/764134110 wait complete. 2024-04-12T19:51:03.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.844+0000 7f5605c00640 1 Processor -- start 2024-04-12T19:51:03.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.845+0000 7f5605c00640 1 -- start start 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55e4004800 0x7f560015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56001f4750 0x7f56001f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5600145450 con 0x7f560015ce20 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5600163000 con 0x7f56001f4750 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f5605c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5600072150 con 0x7f55e4004800 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53316/0 (socket says 172.21.15.32:53316) 2024-04-12T19:51:03.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 -- 172.21.15.32:0/1783939886 learned_addr learned my addr 172.21.15.32:0/1783939886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55ff400640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56001f4750 0x7f56001f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fea00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55e4004800 0x7f560015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 -- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55e4004800 msgr2=0x7f560015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55e4004800 0x7f560015c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 -- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56001f4750 msgr2=0x7f56001f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56001f4750 0x7f56001f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.846+0000 7f55fe000640 1 -- 172.21.15.32:0/1783939886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560015cb80 con 0x7f560015ce20 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f55fe000640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c160 tx=0x7f5600162c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec001ba0 con 0x7f560015ce20 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ec004bf0 con 0x7f560015ce20 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec007070 con 0x7f560015ce20 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560015dca0 con 0x7f560015ce20 2024-04-12T19:51:03.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.847+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001f5610 con 0x7f560015ce20 2024-04-12T19:51:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.849+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600068c80 con 0x7f560015ce20 2024-04-12T19:51:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.849+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55ec007210 con 0x7f560015ce20 2024-04-12T19:51:03.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.851+0000 7f55f6a00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 0x7f55e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:03.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.851+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55ec0ad250 con 0x7f560015ce20 2024-04-12T19:51:03.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.851+0000 7f55fea00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 0x7f55e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:03.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.852+0000 7f55fea00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 0x7f55e006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55f00098b0 tx=0x7f55f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:03.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:03.856+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f560015cb80 con 0x7f560015ce20 2024-04-12T19:51:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.062+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f560006f8d0 con 0x7f560015ce20 2024-04-12T19:51:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.063+0000 7f55f6a00640 1 -- 172.21.15.32:0/1783939886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f55ec07b5e0 con 0x7f560015ce20 2024-04-12T19:51:04.063 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:04.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.065+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 msgr2=0x7f55e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.065+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 0x7f55e006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55f00098b0 tx=0x7f55f0008040 comp rx=0 tx=0).stop 2024-04-12T19:51:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 msgr2=0x7f560015d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c160 tx=0x7f5600162c10 comp rx=0 tx=0).stop 2024-04-12T19:51:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 shutdown_connections 2024-04-12T19:51:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f55e006c8f0 0x7f55e006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56001f4750 0x7f56001f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560015ce20 0x7f560015d2a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 --2- 172.21.15.32:0/1783939886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55e4004800 0x7f560015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 >> 172.21.15.32:0/1783939886 conn(0x7f56000b2460 msgr2=0x7f56000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 shutdown_connections 2024-04-12T19:51:04.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.066+0000 7f5605c00640 1 -- 172.21.15.32:0/1783939886 wait complete. 2024-04-12T19:51:04.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.314+0000 7f54aca00640 1 Processor -- start 2024-04-12T19:51:04.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.315+0000 7f54aca00640 1 -- start start 2024-04-12T19:51:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.316+0000 7f54aca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54a8162730 0x7f54a80b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.316+0000 7f54aca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54a80d2b50 0x7f54a80d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.316+0000 7f54aca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a8162dd0 con 0x7f54a815c680 2024-04-12T19:51:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.316+0000 7f54aca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f54a8162b90 con 0x7f54a8162730 2024-04-12T19:51:04.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.316+0000 7f54aca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f54a81630b0 con 0x7f54a80d2b50 2024-04-12T19:51:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54a8162730 0x7f54a80b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54a8162730 0x7f54a80b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43850/0 (socket says 172.21.15.32:43850) 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 -- 172.21.15.32:0/3820698288 learned_addr learned my addr 172.21.15.32:0/3820698288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a7e00640 1 --2- 172.21.15.32:0/3820698288 >> v2:172.21.15.169:3300/0 conn(0x7f54a80d2b50 0x7f54a80d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 -- 172.21.15.32:0/3820698288 >> v2:172.21.15.169:3300/0 conn(0x7f54a80d2b50 msgr2=0x7f54a80d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 --2- 172.21.15.32:0/3820698288 >> v2:172.21.15.169:3300/0 conn(0x7f54a80d2b50 0x7f54a80d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 -- 172.21.15.32:0/3820698288 >> v1:172.21.15.169:6789/0 conn(0x7f54a815c680 legacy=0x7f54a815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.317+0000 7f54a6a00640 1 -- 172.21.15.32:0/3820698288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a8163390 con 0x7f54a8162730 2024-04-12T19:51:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.318+0000 7f54a6a00640 1 --2- 172.21.15.32:0/3820698288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a8162730 0x7f54a80b6c70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54a80d1680 tx=0x7f549002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255dd4de510dcbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.319+0000 7f54a6000640 1 -- 172.21.15.32:0/3820698288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549002cb10 con 0x7f54a8162730 2024-04-12T19:51:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.319+0000 7f54a6000640 1 -- 172.21.15.32:0/3820698288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549002ccb0 con 0x7f54a8162730 2024-04-12T19:51:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.319+0000 7f54a6000640 1 -- 172.21.15.32:0/3820698288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549002ce60 con 0x7f54a8162730 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.319+0000 7f54aca00640 1 -- 172.21.15.32:0/3820698288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a8162730 msgr2=0x7f54a80b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.319+0000 7f54aca00640 1 --2- 172.21.15.32:0/3820698288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a8162730 0x7f54a80b6c70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54a80d1680 tx=0x7f549002bf70 comp rx=0 tx=0).stop 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 -- 172.21.15.32:0/3820698288 shutdown_connections 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 --2- 172.21.15.32:0/3820698288 >> v2:172.21.15.169:3300/0 conn(0x7f54a80d2b50 0x7f54a80d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 --2- 172.21.15.32:0/3820698288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a8162730 0x7f54a80b6c70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 -- 172.21.15.32:0/3820698288 >> 172.21.15.32:0/3820698288 conn(0x7f54a8073480 msgr2=0x7f54a8073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 -- 172.21.15.32:0/3820698288 shutdown_connections 2024-04-12T19:51:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.320+0000 7f54aca00640 1 -- 172.21.15.32:0/3820698288 wait complete. 2024-04-12T19:51:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.321+0000 7f54aca00640 1 Processor -- start 2024-04-12T19:51:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 -- start start 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a815c680 0x7f54a815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a8162730 0x7f54a8161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54a815ff60 con 0x7f54a80d2b50 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54a815ca60 con 0x7f54a8162730 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.322+0000 7f54aca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80d1770 con 0x7f54a815c680 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43866/0 (socket says 172.21.15.32:43866) 2024-04-12T19:51:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 -- 172.21.15.32:0/3918227155 learned_addr learned my addr 172.21.15.32:0/3918227155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a6a00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a815c680 0x7f54a815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7e00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a8162730 0x7f54a8161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 -- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a815c680 msgr2=0x7f54a815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a815c680 0x7f54a815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 -- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a8162730 msgr2=0x7f54a8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a8162730 0x7f54a8161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.323+0000 7f54a7400640 1 -- 172.21.15.32:0/3918227155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a815bbb0 con 0x7f54a80d2b50 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f54a7400640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f549800c400 tx=0x7f549800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980052d0 con 0x7f54a80d2b50 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5498005490 con 0x7f54a80d2b50 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498007260 con 0x7f54a80d2b50 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a815ab10 con 0x7f54a80d2b50 2024-04-12T19:51:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.324+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a815b0d0 con 0x7f54a80d2b50 2024-04-12T19:51:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.325+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a80755c0 con 0x7f54a80d2b50 2024-04-12T19:51:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.326+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5498007400 con 0x7f54a80d2b50 2024-04-12T19:51:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.327+0000 7f549f400640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 0x7f548406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.327+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5738+0+0 (secure 0 0 0) 0x7f549808b140 con 0x7f54a80d2b50 2024-04-12T19:51:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.328+0000 7f54a6a00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 0x7f548406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.328+0000 7f54a6a00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 0x7f548406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5490035000 tx=0x7f5490030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.332+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54980594b0 con 0x7f54a80d2b50 2024-04-12T19:51:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.538+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54a806dbe0 con 0x7f54a80d2b50 2024-04-12T19:51:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.539+0000 7f549f400640 1 -- 172.21.15.32:0/3918227155 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f5498058db0 con 0x7f54a80d2b50 2024-04-12T19:51:04.540 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 msgr2=0x7f548406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 0x7f548406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5490035000 tx=0x7f5490030000 comp rx=0 tx=0).stop 2024-04-12T19:51:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 msgr2=0x7f54a814bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f549800c400 tx=0x7f549800c8c0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 shutdown_connections 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f548406c8f0 0x7f548406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54a8162730 0x7f54a8161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54a815c680 0x7f54a815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 --2- 172.21.15.32:0/3918227155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54a80d2b50 0x7f54a814bc80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.542+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 >> 172.21.15.32:0/3918227155 conn(0x7f54a8073480 msgr2=0x7f54a8160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.543+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 shutdown_connections 2024-04-12T19:51:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.543+0000 7f54aca00640 1 -- 172.21.15.32:0/3918227155 wait complete. 2024-04-12T19:51:04.578 INFO:tasks.cephfs_test_runner:Starting test: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-04-12T19:51:04.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.808+0000 7f10e8a00640 1 Processor -- start 2024-04-12T19:51:04.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.808+0000 7f10e8a00640 1 -- start start 2024-04-12T19:51:04.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.810+0000 7f10e8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e40cee00 0x7f10e406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.810+0000 7f10e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e415d2f0 0x7f10e40723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.810+0000 7f10e8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e415dbb0 con 0x7f10e4071f50 2024-04-12T19:51:04.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.810+0000 7f10e8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10e406f4f0 con 0x7f10e40cee00 2024-04-12T19:51:04.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.810+0000 7f10e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40728f0 con 0x7f10e415d2f0 2024-04-12T19:51:04.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e40cee00 0x7f10e406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e415d2f0 0x7f10e40723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e40cee00 0x7f10e406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:53134/0 (socket says 172.21.15.32:53134) 2024-04-12T19:51:04.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 -- 172.21.15.32:0/1786159999 learned_addr learned my addr 172.21.15.32:0/1786159999 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 -- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e415d2f0 msgr2=0x7f10e40723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 --2- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e415d2f0 0x7f10e40723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 -- 172.21.15.32:0/1786159999 >> v1:172.21.15.169:6789/0 conn(0x7f10e4071f50 legacy=0x7f10e415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.811+0000 7f10e2800640 1 -- 172.21.15.32:0/1786159999 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e415d8d0 con 0x7f10e40cee00 2024-04-12T19:51:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.813+0000 7f10e2800640 1 --2- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e406e8c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009870 tx=0x7f10d402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb4492eeb93aa8a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.813+0000 7f10e1400640 1 -- 172.21.15.32:0/1786159999 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4034070 con 0x7f10e40cee00 2024-04-12T19:51:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.813+0000 7f10e1400640 1 -- 172.21.15.32:0/1786159999 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4030840 con 0x7f10e40cee00 2024-04-12T19:51:04.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.813+0000 7f10e1400640 1 -- 172.21.15.32:0/1786159999 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4030c10 con 0x7f10e40cee00 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 -- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 msgr2=0x7f10e406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 --2- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e406e8c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009870 tx=0x7f10d402b980 comp rx=0 tx=0).stop 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 -- 172.21.15.32:0/1786159999 shutdown_connections 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 --2- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e415d2f0 0x7f10e40723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 --2- 172.21.15.32:0/1786159999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e406e8c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 -- 172.21.15.32:0/1786159999 >> 172.21.15.32:0/1786159999 conn(0x7f10e406b930 msgr2=0x7f10e406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:04.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 -- 172.21.15.32:0/1786159999 shutdown_connections 2024-04-12T19:51:04.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.814+0000 7f10e8a00640 1 -- 172.21.15.32:0/1786159999 wait complete. 2024-04-12T19:51:04.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.816+0000 7f10e8a00640 1 Processor -- start 2024-04-12T19:51:04.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.816+0000 7f10e8a00640 1 -- start start 2024-04-12T19:51:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e4071f50 0x7f10e40b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e41f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406e3a0 con 0x7f10e415d2f0 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406e0c0 con 0x7f10e40cee00 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e415dda0 con 0x7f10e4071f50 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43882/0 (socket says 172.21.15.32:43882) 2024-04-12T19:51:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.817+0000 7f10e3200640 1 -- 172.21.15.32:0/53825790 learned_addr learned my addr 172.21.15.32:0/53825790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e2800640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e4071f50 0x7f10e40b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e1e00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e41f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 -- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e4071f50 msgr2=0x7f10e40b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e4071f50 0x7f10e40b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 -- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 msgr2=0x7f10e41f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:04.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e41f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 -- 172.21.15.32:0/53825790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41f3ae0 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10e3200640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00efc0 tx=0x7f10dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.818+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc018020 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.819+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc014480 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.819+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e40b0740 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.819+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc014870 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.819+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e40b0d00 con 0x7f10e415d2f0 2024-04-12T19:51:04.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.820+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e415cd80 con 0x7f10e415d2f0 2024-04-12T19:51:04.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.821+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10dc005d10 con 0x7f10e415d2f0 2024-04-12T19:51:04.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.822+0000 7f10daa00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:04.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.822+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5738+0+0 (secure 0 0 0) 0x7f10dc08c5d0 con 0x7f10e415d2f0 2024-04-12T19:51:04.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.822+0000 7f10e2800640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:04.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.823+0000 7f10e2800640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10d4036000 tx=0x7f10d403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:04.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:04.827+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10e415cd80 con 0x7f10e415d2f0 2024-04-12T19:51:05.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.039+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10e40755c0 con 0x7f10e415d2f0 2024-04-12T19:51:05.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.041+0000 7f10daa00640 1 -- 172.21.15.32:0/53825790 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f10dc022070 con 0x7f10e415d2f0 2024-04-12T19:51:05.042 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:05.042 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 msgr2=0x7f10c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10d4036000 tx=0x7f10d403b040 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 msgr2=0x7f10e414edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00efc0 tx=0x7f10dc00c490 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 shutdown_connections 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e415d2f0 0x7f10e414edd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e40cee00 0x7f10e41f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 --2- 172.21.15.32:0/53825790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e4071f50 0x7f10e40b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 >> 172.21.15.32:0/53825790 conn(0x7f10e406b930 msgr2=0x7f10e41601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 shutdown_connections 2024-04-12T19:51:05.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.044+0000 7f10e8a00640 1 -- 172.21.15.32:0/53825790 wait complete. 2024-04-12T19:51:05.063 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:05.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.291+0000 7fec70600640 1 Processor -- start 2024-04-12T19:51:05.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.291+0000 7fec70600640 1 -- start start 2024-04-12T19:51:05.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.293+0000 7fec70600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec6815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.293+0000 7fec70600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6815ffe0 con 0x7fec68145070 2024-04-12T19:51:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.293+0000 7fec70600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec68162a40 con 0x7fec68163520 2024-04-12T19:51:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.293+0000 7fec70600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec680afde0 con 0x7fec68072860 2024-04-12T19:51:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec6dc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec68145070 0x7fec68072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33236/0 (socket says 172.21.15.32:33236) 2024-04-12T19:51:05.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec6dc00640 1 -- 172.21.15.32:0/2120704118 learned_addr learned my addr 172.21.15.32:0/2120704118 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599732521 0 0) 0x7fec6815ffe0 con 0x7fec68145070 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec6d200640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec6815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec50003520 con 0x7fec68145070 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3237070958 0 0) 0x7fec50003520 con 0x7fec68145070 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 msgr2=0x7fec6815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.294+0000 7fec67e00640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec6815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 >> v1:172.21.15.169:6789/0 conn(0x7fec68163520 legacy=0x7fec6815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec68162d20 con 0x7fec68145070 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fec5c001670 con 0x7fec68145070 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 >> v1:172.21.15.32:6789/0 conn(0x7fec68145070 legacy=0x7fec68072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec50004060 0x7fec50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec680afde0 con 0x7fec50004d40 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec5000bca0 con 0x7fec50004060 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec5000bea0 con 0x7fec68072860 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec6e600640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec6d200640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec50004060 0x7fec50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.295+0000 7fec6dc00640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.296+0000 7fec6e600640 1 -- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 msgr2=0x7fec500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.296+0000 7fec6e600640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec68072860 0x7fec500039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.296+0000 7fec6e600640 1 -- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec50004060 msgr2=0x7fec50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.296+0000 7fec6e600640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec50004060 0x7fec50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.296+0000 7fec6e600640 1 -- 172.21.15.32:0/2120704118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec50003520 con 0x7fec50004d40 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.297+0000 7fec6e600640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec500051c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fec60001e50 tx=0x7fec50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4089d9593212a786 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.297+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60001a70 con 0x7fec50004d40 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.297+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60003b70 con 0x7fec50004d40 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.297+0000 7fec67e00640 1 -- 172.21.15.32:0/2120704118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec600050d0 con 0x7fec50004d40 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 -- 172.21.15.32:0/2120704118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 msgr2=0x7fec500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 --2- 172.21.15.32:0/2120704118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec500051c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fec60001e50 tx=0x7fec50005830 comp rx=0 tx=0).stop 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec6e600640 1 -- 172.21.15.32:0/2120704118 reap_dead start 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 -- 172.21.15.32:0/2120704118 shutdown_connections 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 -- 172.21.15.32:0/2120704118 >> 172.21.15.32:0/2120704118 conn(0x7fec680b2460 msgr2=0x7fec680b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 -- 172.21.15.32:0/2120704118 shutdown_connections 2024-04-12T19:51:05.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.298+0000 7fec70600640 1 -- 172.21.15.32:0/2120704118 wait complete. 2024-04-12T19:51:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.300+0000 7fec70600640 1 Processor -- start 2024-04-12T19:51:05.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.300+0000 7fec70600640 1 -- start start 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec681f8c80 0x7fec68146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec68146ea0 0x7fec681455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec68145450 con 0x7fec50004d40 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec681627f0 con 0x7fec68146ea0 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec70600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec6815f1a0 con 0x7fec681f8c80 2024-04-12T19:51:05.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43902/0 (socket says 172.21.15.32:43902) 2024-04-12T19:51:05.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 -- 172.21.15.32:0/2206848539 learned_addr learned my addr 172.21.15.32:0/2206848539 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6d200640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec681f8c80 0x7fec68146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6e600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec68146ea0 0x7fec681455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 -- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec681f8c80 msgr2=0x7fec68146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec681f8c80 0x7fec68146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 -- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec68146ea0 msgr2=0x7fec681455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec68146ea0 0x7fec681455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.301+0000 7fec6dc00640 1 -- 172.21.15.32:0/2206848539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec681588a0 con 0x7fec50004d40 2024-04-12T19:51:05.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec6dc00640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d7c0 tx=0x7fec5c00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c019070 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5c0074e0 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004a00 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec68146300 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.302+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec680ce130 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.304+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec681472b0 con 0x7fec50004d40 2024-04-12T19:51:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.305+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec5c004ba0 con 0x7fec50004d40 2024-04-12T19:51:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.306+0000 7fec65600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 0x7fec4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.306+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fec5c08b550 con 0x7fec50004d40 2024-04-12T19:51:05.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.306+0000 7fec6d200640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 0x7fec4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.307+0000 7fec6d200640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 0x7fec4806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec68142010 tx=0x7fec54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.311+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec681472b0 con 0x7fec50004d40 2024-04-12T19:51:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.516+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec680b2cb0 con 0x7fec50004d40 2024-04-12T19:51:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.517+0000 7fec65600640 1 -- 172.21.15.32:0/2206848539 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fec5c0599f0 con 0x7fec50004d40 2024-04-12T19:51:05.517 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 msgr2=0x7fec4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 0x7fec4806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec68142010 tx=0x7fec54008040 comp rx=0 tx=0).stop 2024-04-12T19:51:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 msgr2=0x7fec68158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d7c0 tx=0x7fec5c00dc80 comp rx=0 tx=0).stop 2024-04-12T19:51:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 shutdown_connections 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec4806c8f0 0x7fec4806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec68146ea0 0x7fec681455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec681f8c80 0x7fec68146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 --2- 172.21.15.32:0/2206848539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec50004d40 0x7fec68158360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 >> 172.21.15.32:0/2206848539 conn(0x7fec680b2460 msgr2=0x7fec680adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 shutdown_connections 2024-04-12T19:51:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.520+0000 7fec70600640 1 -- 172.21.15.32:0/2206848539 wait complete. 2024-04-12T19:51:05.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.778+0000 7fe295e00640 1 Processor -- start 2024-04-12T19:51:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.778+0000 7fe295e00640 1 -- start start 2024-04-12T19:51:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe295e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2900725f0 0x7fe2900ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe295e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 0x7fe29006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe295e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2900cd2d0 con 0x7fe2901461c0 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe295e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe2900cf1d0 con 0x7fe2900725f0 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe295e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe2900cd080 con 0x7fe2900ce9e0 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe28fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2900725f0 0x7fe2900ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe294a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2901461c0 0x7fe290071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33248/0 (socket says 172.21.15.32:33248) 2024-04-12T19:51:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.780+0000 7fe294a00640 1 -- 172.21.15.32:0/1509256047 learned_addr learned my addr 172.21.15.32:0/1509256047 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe28fe00640 1 -- 172.21.15.32:0/1509256047 >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 msgr2=0x7fe29006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe295400640 1 --2- 172.21.15.32:0/1509256047 >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 0x7fe29006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe28fe00640 1 --2- 172.21.15.32:0/1509256047 >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 0x7fe29006df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe28fe00640 1 -- 172.21.15.32:0/1509256047 >> v1:172.21.15.32:6789/0 conn(0x7fe2901461c0 legacy=0x7fe290071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe28fe00640 1 -- 172.21.15.32:0/1509256047 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2900cd5b0 con 0x7fe2900725f0 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe28f400640 1 -- 172.21.15.32:0/1509256047 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106851261 0 0) 0x7fe2900cd2d0 con 0x7fe2901461c0 2024-04-12T19:51:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.781+0000 7fe295400640 1 --2- 172.21.15.32:0/1509256047 >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 0x7fe29006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.782+0000 7fe28fe00640 1 --2- 172.21.15.32:0/1509256047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900725f0 0x7fe2900ce4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b60e0 tx=0x7fe27c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5337c11595ed6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe28f400640 1 -- 172.21.15.32:0/1509256047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c035020 con 0x7fe2900725f0 2024-04-12T19:51:05.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe28f400640 1 -- 172.21.15.32:0/1509256047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c031830 con 0x7fe2900725f0 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe28f400640 1 -- 172.21.15.32:0/1509256047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031c00 con 0x7fe2900725f0 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 -- 172.21.15.32:0/1509256047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900725f0 msgr2=0x7fe2900ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 --2- 172.21.15.32:0/1509256047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900725f0 0x7fe2900ce4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b60e0 tx=0x7fe27c02b8d0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 -- 172.21.15.32:0/1509256047 shutdown_connections 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 --2- 172.21.15.32:0/1509256047 >> v2:172.21.15.169:3300/0 conn(0x7fe2900ce9e0 0x7fe29006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 --2- 172.21.15.32:0/1509256047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900725f0 0x7fe2900ce4a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 -- 172.21.15.32:0/1509256047 >> 172.21.15.32:0/1509256047 conn(0x7fe29006b930 msgr2=0x7fe29006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.783+0000 7fe295e00640 1 -- 172.21.15.32:0/1509256047 shutdown_connections 2024-04-12T19:51:05.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.784+0000 7fe295e00640 1 -- 172.21.15.32:0/1509256047 wait complete. 2024-04-12T19:51:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.785+0000 7fe295e00640 1 Processor -- start 2024-04-12T19:51:05.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.785+0000 7fe295e00640 1 -- start start 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2900725f0 0x7fe2900b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2901461c0 0x7fe29015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900cee30 con 0x7fe2900ce9e0 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900cdb70 con 0x7fe2900725f0 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.786+0000 7fe295e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2900cefb0 con 0x7fe2901461c0 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43910/0 (socket says 172.21.15.32:43910) 2024-04-12T19:51:05.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 -- 172.21.15.32:0/2484695639 learned_addr learned my addr 172.21.15.32:0/2484695639 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe294a00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2900725f0 0x7fe2900b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe295400640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2901461c0 0x7fe29015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 -- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2901461c0 msgr2=0x7fe29015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2901461c0 0x7fe29015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 -- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2900725f0 msgr2=0x7fe2900b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2900725f0 0x7fe2900b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.787+0000 7fe28fe00640 1 -- 172.21.15.32:0/2484695639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2900b4b30 con 0x7fe2900ce9e0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe28fe00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c035050 con 0x7fe2900ce9e0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c004230 con 0x7fe2900ce9e0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c03b800 con 0x7fe2900ce9e0 2024-04-12T19:51:05.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2900af330 con 0x7fe2900ce9e0 2024-04-12T19:51:05.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.788+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe290159f40 con 0x7fe2900ce9e0 2024-04-12T19:51:05.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.789+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29015db40 con 0x7fe2900ce9e0 2024-04-12T19:51:05.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.790+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe27c038020 con 0x7fe2900ce9e0 2024-04-12T19:51:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.791+0000 7fe28cc00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 0x7fe27006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.791+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fe27c0ada40 con 0x7fe2900ce9e0 2024-04-12T19:51:05.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.791+0000 7fe294a00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 0x7fe27006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:05.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.792+0000 7fe294a00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 0x7fe27006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe284000f60 tx=0x7fe284009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:05.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:05.796+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe29015db40 con 0x7fe2900ce9e0 2024-04-12T19:51:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.002+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2900b4d60 con 0x7fe2900ce9e0 2024-04-12T19:51:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.002+0000 7fe28cc00640 1 -- 172.21.15.32:0/2484695639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fe27c040070 con 0x7fe2900ce9e0 2024-04-12T19:51:06.003 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 msgr2=0x7fe27006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 0x7fe27006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe284000f60 tx=0x7fe284009040 comp rx=0 tx=0).stop 2024-04-12T19:51:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 msgr2=0x7fe29015a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 shutdown_connections 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe27006c8f0 0x7fe27006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2901461c0 0x7fe29015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2900ce9e0 0x7fe29015a8e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 --2- 172.21.15.32:0/2484695639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2900725f0 0x7fe2900b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 >> 172.21.15.32:0/2484695639 conn(0x7fe29006b930 msgr2=0x7fe2901425f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 shutdown_connections 2024-04-12T19:51:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.005+0000 7fe295e00640 1 -- 172.21.15.32:0/2484695639 wait complete. 2024-04-12T19:51:06.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.248+0000 7f8921000640 1 Processor -- start 2024-04-12T19:51:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.248+0000 7f8921000640 1 -- start start 2024-04-12T19:51:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8921000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8921000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f891c072690 0x7f891c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8921000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f891c15ffe0 con 0x7f891c163520 2024-04-12T19:51:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8921000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f891c0afde0 con 0x7f891c072690 2024-04-12T19:51:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8921000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f891c1627f0 con 0x7f891c145070 2024-04-12T19:51:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8913e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f891c163520 0x7f891c15f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33252/0 (socket says 172.21.15.32:33252) 2024-04-12T19:51:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f8913e00640 1 -- 172.21.15.32:0/706890704 learned_addr learned my addr 172.21.15.32:0/706890704 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.250+0000 7f891a800640 1 --2- 172.21.15.32:0/706890704 >> v2:172.21.15.32:3300/0 conn(0x7f891c072690 0x7f891c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f8919e00640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f8919400640 1 -- 172.21.15.32:0/706890704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761173599 0 0) 0x7f891c15ffe0 con 0x7f891c163520 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f8919400640 1 -- 172.21.15.32:0/706890704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8900003520 con 0x7f891c163520 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f891a800640 1 -- 172.21.15.32:0/706890704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 msgr2=0x7f891c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f891a800640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f891a800640 1 -- 172.21.15.32:0/706890704 >> v1:172.21.15.32:6789/0 conn(0x7f891c163520 legacy=0x7f891c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.251+0000 7f891a800640 1 -- 172.21.15.32:0/706890704 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c162a40 con 0x7f891c072690 2024-04-12T19:51:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.252+0000 7f891a800640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c072690 0x7f891c15f8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8914009d90 tx=0x7f891402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dbdfc92b154165 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.252+0000 7f8919400640 1 -- 172.21.15.32:0/706890704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914039070 con 0x7f891c072690 2024-04-12T19:51:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8919400640 1 -- 172.21.15.32:0/706890704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891402cbc0 con 0x7f891c072690 2024-04-12T19:51:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8919400640 1 -- 172.21.15.32:0/706890704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914034440 con 0x7f891c072690 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 -- 172.21.15.32:0/706890704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c072690 msgr2=0x7f891c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c072690 0x7f891c15f8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8914009d90 tx=0x7f891402be10 comp rx=0 tx=0).stop 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 -- 172.21.15.32:0/706890704 shutdown_connections 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c072690 0x7f891c15f8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 --2- 172.21.15.32:0/706890704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.253+0000 7f8921000640 1 -- 172.21.15.32:0/706890704 >> 172.21.15.32:0/706890704 conn(0x7f891c0b2460 msgr2=0x7f891c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:06.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.254+0000 7f8921000640 1 -- 172.21.15.32:0/706890704 shutdown_connections 2024-04-12T19:51:06.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.254+0000 7f8921000640 1 -- 172.21.15.32:0/706890704 wait complete. 2024-04-12T19:51:06.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.255+0000 7f8921000640 1 Processor -- start 2024-04-12T19:51:06.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.255+0000 7f8921000640 1 -- start start 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 0x7f891c142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c1455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f891c145450 con 0x7f891c163520 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f891c163000 con 0x7f891c072690 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.256+0000 7f8921000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f891c0aedd0 con 0x7f891c145070 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43920/0 (socket says 172.21.15.32:43920) 2024-04-12T19:51:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 -- 172.21.15.32:0/1329210637 learned_addr learned my addr 172.21.15.32:0/1329210637 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:06.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f8919e00640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 0x7f891c142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f8913e00640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c1455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 -- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 msgr2=0x7f891c1455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c1455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 -- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 msgr2=0x7f891c142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 0x7f891c142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f891a800640 1 -- 172.21.15.32:0/1329210637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c145f90 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.257+0000 7f8919e00640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 0x7f891c142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f891a800640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8914009d90 tx=0x7f8914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914039050 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914045070 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c146270 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914009450 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.258+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c146830 con 0x7f891c163520 2024-04-12T19:51:06.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.259+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c068c80 con 0x7f891c163520 2024-04-12T19:51:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.260+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f891403e850 con 0x7f891c163520 2024-04-12T19:51:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.262+0000 7f8912000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 0x7f88f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.262+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f89140ae190 con 0x7f891c163520 2024-04-12T19:51:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.262+0000 7f8919e00640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 0x7f88f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.263+0000 7f8919e00640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 0x7f88f806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f890c009ac0 tx=0x7f890c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.266+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f891404e020 con 0x7f891c163520 2024-04-12T19:51:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.470+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f891c143bf0 con 0x7f891c163520 2024-04-12T19:51:06.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.471+0000 7f8912000640 1 -- 172.21.15.32:0/1329210637 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f8914032e20 con 0x7f891c163520 2024-04-12T19:51:06.471 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.473+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 msgr2=0x7f88f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.473+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 0x7f88f806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f890c009ac0 tx=0x7f890c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.473+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 msgr2=0x7f891c147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.473+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8914009d90 tx=0x7f8914004a40 comp rx=0 tx=0).stop 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 shutdown_connections 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f88f806c920 0x7f88f806ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f891c163520 0x7f891c147030 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c145070 0x7f891c1455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 --2- 172.21.15.32:0/1329210637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c072690 0x7f891c142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 >> 172.21.15.32:0/1329210637 conn(0x7f891c0b2460 msgr2=0x7f891c0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 shutdown_connections 2024-04-12T19:51:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.474+0000 7f8921000640 1 -- 172.21.15.32:0/1329210637 wait complete. 2024-04-12T19:51:06.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_flush.TestFlush.test_flush' 2024-04-12T19:51:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.714+0000 7f6ce7000640 1 Processor -- start 2024-04-12T19:51:06.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.714+0000 7f6ce7000640 1 -- start start 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.715+0000 7f6ce7000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ce00cedf0 0x7f6ce009b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.715+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009b1e0 0x7f6ce0160f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6ce7000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce00c95e0 con 0x7f6ce009ae00 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6ce7000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce0160520 con 0x7f6ce00cedf0 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6ce7000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce00d0320 con 0x7f6ce009b1e0 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6cdf400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ce00cedf0 0x7f6ce009b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6cdfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ce009ae00 0x7f6ce00cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33266/0 (socket says 172.21.15.32:33266) 2024-04-12T19:51:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6cdfe00640 1 -- 172.21.15.32:0/798077565 learned_addr learned my addr 172.21.15.32:0/798077565 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:06.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.716+0000 7f6ce5000640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009b1e0 0x7f6ce0160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdea00640 1 -- 172.21.15.32:0/798077565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838458446 0 0) 0x7f6ce00c95e0 con 0x7f6ce009ae00 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdea00640 1 -- 172.21.15.32:0/798077565 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cc4003520 con 0x7f6ce009ae00 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdf400640 1 -- 172.21.15.32:0/798077565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009b1e0 msgr2=0x7f6ce0160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdf400640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009b1e0 0x7f6ce0160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdf400640 1 -- 172.21.15.32:0/798077565 >> v1:172.21.15.32:6789/0 conn(0x7f6ce009ae00 legacy=0x7f6ce00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.717+0000 7f6cdf400640 1 -- 172.21.15.32:0/798077565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce00d05e0 con 0x7f6ce00cedf0 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.718+0000 7f6cdf400640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce009b970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f31bf51795ce7a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6cdea00640 1 -- 172.21.15.32:0/798077565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc039070 con 0x7f6ce00cedf0 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6cdea00640 1 -- 172.21.15.32:0/798077565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc02cbc0 con 0x7f6ce00cedf0 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6cdea00640 1 -- 172.21.15.32:0/798077565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc034440 con 0x7f6ce00cedf0 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 -- 172.21.15.32:0/798077565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 msgr2=0x7f6ce009b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce009b970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc02be10 comp rx=0 tx=0).stop 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 -- 172.21.15.32:0/798077565 shutdown_connections 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009b1e0 0x7f6ce0160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 --2- 172.21.15.32:0/798077565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce009b970 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 -- 172.21.15.32:0/798077565 >> 172.21.15.32:0/798077565 conn(0x7f6ce00d2750 msgr2=0x7f6ce00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.719+0000 7f6ce7000640 1 -- 172.21.15.32:0/798077565 shutdown_connections 2024-04-12T19:51:06.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.720+0000 7f6ce7000640 1 -- 172.21.15.32:0/798077565 wait complete. 2024-04-12T19:51:06.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.721+0000 7f6ce7000640 1 Processor -- start 2024-04-12T19:51:06.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.721+0000 7f6ce7000640 1 -- start start 2024-04-12T19:51:06.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009ae00 0x7f6ce00b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce009b1e0 0x7f6ce0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce015d900 con 0x7f6ce00cedf0 2024-04-12T19:51:06.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce0160770 con 0x7f6ce009b1e0 2024-04-12T19:51:06.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.722+0000 7f6ce7000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce015da80 con 0x7f6ce009ae00 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43932/0 (socket says 172.21.15.32:43932) 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 -- 172.21.15.32:0/3947084990 learned_addr learned my addr 172.21.15.32:0/3947084990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6cdfe00640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009ae00 0x7f6ce00b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6cdf400640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce009b1e0 0x7f6ce0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 -- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009ae00 msgr2=0x7f6ce00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009ae00 0x7f6ce00b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 -- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce009b1e0 msgr2=0x7f6ce0153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce009b1e0 0x7f6ce0153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.723+0000 7f6ce5000640 1 -- 172.21.15.32:0/3947084990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce006d140 con 0x7f6ce00cedf0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6ce5000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d800 tx=0x7f6cd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800b6c0 con 0x7f6ce00cedf0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd8019070 con 0x7f6ce00cedf0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800bac0 con 0x7f6ce00cedf0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce006d420 con 0x7f6ce00cedf0 2024-04-12T19:51:06.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.724+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce00b5780 con 0x7f6ce00cedf0 2024-04-12T19:51:06.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.726+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce0068c80 con 0x7f6ce00cedf0 2024-04-12T19:51:06.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.726+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cd800bc60 con 0x7f6ce00cedf0 2024-04-12T19:51:06.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.728+0000 7f6cd7e00640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:06.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.728+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f6cd808bfe0 con 0x7f6ce00cedf0 2024-04-12T19:51:06.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.728+0000 7f6cdfe00640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:06.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.729+0000 7f6cdfe00640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 0x7f6cc006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005880 tx=0x7f6cd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:06.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:06.733+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cd805a3f0 con 0x7f6ce00cedf0 2024-04-12T19:51:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.018+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) -- 0x7f6ce00ccb60 con 0x7f6ce00cedf0 2024-04-12T19:51:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.925+0000 7f6cd7e00640 1 -- 172.21.15.32:0/3947084990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v63) ==== 126+0+0 (secure 0 0 0) 0x7f6cd8059cf0 con 0x7f6ce00cedf0 2024-04-12T19:51:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 msgr2=0x7f6cc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 0x7f6cc006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005880 tx=0x7f6cd0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 msgr2=0x7f6ce006df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d800 tx=0x7f6cd800dcc0 comp rx=0 tx=0).stop 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 shutdown_connections 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6cc006c8f0 0x7f6cc006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce00cedf0 0x7f6ce006df00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce009b1e0 0x7f6ce0153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 --2- 172.21.15.32:0/3947084990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ce009ae00 0x7f6ce00b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 >> 172.21.15.32:0/3947084990 conn(0x7f6ce00d2750 msgr2=0x7f6ce0071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.929+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 shutdown_connections 2024-04-12T19:51:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:07.930+0000 7f6ce7000640 1 -- 172.21.15.32:0/3947084990 wait complete. 2024-04-12T19:51:07.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-12T19:51:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.184+0000 7f6e98e00640 1 Processor -- start 2024-04-12T19:51:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.185+0000 7f6e98e00640 1 -- start start 2024-04-12T19:51:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.186+0000 7f6e98e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e941461c0 0x7f6e94071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.186+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e94072420 0x7f6e940ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.186+0000 7f6e98e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e940cd2d0 con 0x7f6e940ce9e0 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.186+0000 7f6e98e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6e940cf1d0 con 0x7f6e941461c0 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.186+0000 7f6e98e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e94072880 con 0x7f6e94072420 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.187+0000 7f6e92a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e94072420 0x7f6e940ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.187+0000 7f6e93400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e941461c0 0x7f6e94071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.187+0000 7f6e93400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e941461c0 0x7f6e94071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:53204/0 (socket says 172.21.15.32:53204) 2024-04-12T19:51:08.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.187+0000 7f6e93400640 1 -- 172.21.15.32:0/524618836 learned_addr learned my addr 172.21.15.32:0/524618836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:08.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.188+0000 7f6e93400640 1 -- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e94072420 msgr2=0x7f6e940ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:08.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.188+0000 7f6e93400640 1 --2- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e94072420 0x7f6e940ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.188+0000 7f6e93400640 1 -- 172.21.15.32:0/524618836 >> v1:172.21.15.169:6789/0 conn(0x7f6e940ce9e0 legacy=0x7f6e94160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:08.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.188+0000 7f6e93400640 1 -- 172.21.15.32:0/524618836 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e940cd5b0 con 0x7f6e941461c0 2024-04-12T19:51:08.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.189+0000 7f6e93400640 1 --2- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e941461c0 0x7f6e94071ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009870 tx=0x7f6e8402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e10ccf5c9efc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:08.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.190+0000 7f6e92000640 1 -- 172.21.15.32:0/524618836 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84034070 con 0x7f6e941461c0 2024-04-12T19:51:08.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.190+0000 7f6e92000640 1 -- 172.21.15.32:0/524618836 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e84030840 con 0x7f6e941461c0 2024-04-12T19:51:08.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.190+0000 7f6e92000640 1 -- 172.21.15.32:0/524618836 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84030c10 con 0x7f6e941461c0 2024-04-12T19:51:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.190+0000 7f6e98e00640 1 -- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e941461c0 msgr2=0x7f6e94071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:08.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.190+0000 7f6e98e00640 1 --2- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e941461c0 0x7f6e94071ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009870 tx=0x7f6e8402b980 comp rx=0 tx=0).stop 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 -- 172.21.15.32:0/524618836 shutdown_connections 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 --2- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e94072420 0x7f6e940ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 --2- 172.21.15.32:0/524618836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e941461c0 0x7f6e94071ee0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 -- 172.21.15.32:0/524618836 >> 172.21.15.32:0/524618836 conn(0x7f6e9406b930 msgr2=0x7f6e9406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 -- 172.21.15.32:0/524618836 shutdown_connections 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.191+0000 7f6e98e00640 1 -- 172.21.15.32:0/524618836 wait complete. 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.192+0000 7f6e98e00640 1 Processor -- start 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.193+0000 7f6e98e00640 1 -- start start 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e94072420 0x7f6e94155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e940ce9e0 0x7f6e94159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940cdb70 con 0x7f6e941461c0 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940cd080 con 0x7f6e94072420 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e98e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940cee70 con 0x7f6e940ce9e0 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e93e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e93e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43944/0 (socket says 172.21.15.32:43944) 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e93e00640 1 -- 172.21.15.32:0/2778209804 learned_addr learned my addr 172.21.15.32:0/2778209804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e92a00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e940ce9e0 0x7f6e94159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.194+0000 7f6e93400640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e94072420 0x7f6e94155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 -- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e940ce9e0 msgr2=0x7f6e94159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e940ce9e0 0x7f6e94159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 -- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e94072420 msgr2=0x7f6e94155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:08.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e94072420 0x7f6e94155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 -- 172.21.15.32:0/2778209804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e941576f0 con 0x7f6e941461c0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e93e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c7a0 tx=0x7f6e8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c003b70 con 0x7f6e941461c0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e8c003d30 con 0x7f6e941461c0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00aa90 con 0x7f6e941461c0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e941579d0 con 0x7f6e941461c0 2024-04-12T19:51:08.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.195+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e94157f90 con 0x7f6e941461c0 2024-04-12T19:51:08.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.197+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e940755c0 con 0x7f6e941461c0 2024-04-12T19:51:08.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.198+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e8c00ac30 con 0x7f6e941461c0 2024-04-12T19:51:08.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.199+0000 7f6e8b400640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 0x7f6e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:08.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.199+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f6e8c08bba0 con 0x7f6e941461c0 2024-04-12T19:51:08.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.199+0000 7f6e93400640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 0x7f6e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.200+0000 7f6e93400640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 0x7f6e7006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e84036000 tx=0x7f6e8403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:08.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.204+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e8c059f10 con 0x7f6e941461c0 2024-04-12T19:51:08.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.414+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6e94072e40 con 0x7f6e941461c0 2024-04-12T19:51:08.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.421+0000 7f6e8b400640 1 -- 172.21.15.32:0/2778209804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f6e8c059810 con 0x7f6e941461c0 2024-04-12T19:51:08.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 msgr2=0x7f6e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:08.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 0x7f6e7006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6e84036000 tx=0x7f6e8403b040 comp rx=0 tx=0).stop 2024-04-12T19:51:08.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 msgr2=0x7f6e9415a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:08.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c7a0 tx=0x7f6e8c00cc60 comp rx=0 tx=0).stop 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 shutdown_connections 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e7006c8f0 0x7f6e7006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e941461c0 0x7f6e9415a4f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e940ce9e0 0x7f6e94159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 --2- 172.21.15.32:0/2778209804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e94072420 0x7f6e94155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 >> 172.21.15.32:0/2778209804 conn(0x7f6e9406b930 msgr2=0x7f6e941444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 shutdown_connections 2024-04-12T19:51:08.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:08.424+0000 7f6e98e00640 1 -- 172.21.15.32:0/2778209804 wait complete. 2024-04-12T19:51:08.444 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-12T19:51:08.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-12T19:51:08.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-12T19:51:08.701 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-12T19:51:08.701 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:08.702 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:08.762 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:08.763 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:08.816 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:08.816 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:08.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:09.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.098+0000 7f4e04a00640 1 Processor -- start 2024-04-12T19:51:09.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.099+0000 7f4e04a00640 1 -- start start 2024-04-12T19:51:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.100+0000 7f4e04a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e00072660 0x7f4e0006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.100+0000 7f4e04a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0015dbb0 con 0x7f4e000cee00 2024-04-12T19:51:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.100+0000 7f4e04a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0015d8d0 con 0x7f4e0015cbe0 2024-04-12T19:51:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.100+0000 7f4e04a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4e0006f4f0 con 0x7f4e00072660 2024-04-12T19:51:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.101+0000 7f4dff400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e000cee00 0x7f4e00071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33276/0 (socket says 172.21.15.32:33276) 2024-04-12T19:51:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.101+0000 7f4dff400640 1 -- 172.21.15.32:0/1862219216 learned_addr learned my addr 172.21.15.32:0/1862219216 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.101+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424843621 0 0) 0x7f4e0015dbb0 con 0x7f4e000cee00 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.101+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de4003520 con 0x7f4e000cee00 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.101+0000 7f4dfea00640 1 --2- 172.21.15.32:0/1862219216 >> v2:172.21.15.169:3300/0 conn(0x7f4e00072660 0x7f4e0006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184353778 0 0) 0x7f4de4003520 con 0x7f4e000cee00 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 >> v2:172.21.15.169:3300/0 conn(0x7f4e00072660 msgr2=0x7f4e0006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 --2- 172.21.15.32:0/1862219216 >> v2:172.21.15.169:3300/0 conn(0x7f4e00072660 0x7f4e0006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 >> v1:172.21.15.169:6789/0 conn(0x7f4e0015cbe0 legacy=0x7f4e0015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0006dde0 con 0x7f4e000cee00 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f4df0001670 con 0x7f4e000cee00 2024-04-12T19:51:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 >> v1:172.21.15.32:6789/0 conn(0x7f4e000cee00 legacy=0x7f4e00071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40039b0 0x7f4de4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4004800 0x7f4de4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 0x7f4de400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4de400c0c0 con 0x7f4de40051c0 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4de400c270 con 0x7f4de4004800 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4de400c470 con 0x7f4de40039b0 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dffe00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 0x7f4de400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dff400640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40039b0 0x7f4de4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.102+0000 7f4dfea00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4004800 0x7f4de4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 -- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40039b0 msgr2=0x7f4de4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40039b0 0x7f4de4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 -- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4004800 msgr2=0x7f4de4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4004800 0x7f4de4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 -- 172.21.15.32:0/1862219216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de4003520 con 0x7f4de40051c0 2024-04-12T19:51:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.103+0000 7f4dffe00640 1 -- 172.21.15.32:0/1862219216 reap_dead start 2024-04-12T19:51:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4dffe00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 0x7f4de400b9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4df8004480 tx=0x7f4de4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b68da99f43f879 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8034070 con 0x7f4de40051c0 2024-04-12T19:51:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8030400 con 0x7f4de40051c0 2024-04-12T19:51:09.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4dfe000640 1 -- 172.21.15.32:0/1862219216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8004080 con 0x7f4de40051c0 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4e04a00640 1 -- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 msgr2=0x7f4de400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.104+0000 7f4e04a00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 0x7f4de400b9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4df8004480 tx=0x7f4de4005830 comp rx=0 tx=0).stop 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.105+0000 7f4e04a00640 1 -- 172.21.15.32:0/1862219216 shutdown_connections 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.105+0000 7f4e04a00640 1 --2- 172.21.15.32:0/1862219216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de40051c0 0x7f4de400b9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.105+0000 7f4e04a00640 1 -- 172.21.15.32:0/1862219216 >> 172.21.15.32:0/1862219216 conn(0x7f4e0006b930 msgr2=0x7f4e0006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.105+0000 7f4e04a00640 1 -- 172.21.15.32:0/1862219216 shutdown_connections 2024-04-12T19:51:09.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.105+0000 7f4e04a00640 1 -- 172.21.15.32:0/1862219216 wait complete. 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.107+0000 7f4e04a00640 1 Processor -- start 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.107+0000 7f4e04a00640 1 -- start start 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40051c0 0x7f4e00152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e00160c20 0x7f4e001f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0006e8c0 con 0x7f4e001f36d0 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0006e3a0 con 0x7f4e00160c20 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4e04a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0006ea40 con 0x7f4de40051c0 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4dffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4dffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43966/0 (socket says 172.21.15.32:43966) 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4dffe00640 1 -- 172.21.15.32:0/2988875553 learned_addr learned my addr 172.21.15.32:0/2988875553 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:09.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4dff400640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40051c0 0x7f4e00152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.108+0000 7f4dfea00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e00160c20 0x7f4e001f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 -- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40051c0 msgr2=0x7f4e00152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40051c0 0x7f4e00152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 -- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e00160c20 msgr2=0x7f4e001f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e00160c20 0x7f4e001f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 -- 172.21.15.32:0/2988875553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e000b5310 con 0x7f4e001f36d0 2024-04-12T19:51:09.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4dffe00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4df802bf60 tx=0x7f4df8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8034040 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df8006d50 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.109+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df80398a0 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.110+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e000b55f0 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.110+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e001f3b50 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.111+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e001561f0 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.112+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4df8037070 con 0x7f4e001f36d0 2024-04-12T19:51:09.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.113+0000 7f4df7400640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 0x7f4de006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.113+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f4df80ad2b0 con 0x7f4e001f36d0 2024-04-12T19:51:09.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.113+0000 7f4dff400640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 0x7f4de006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.114+0000 7f4dff400640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 0x7f4de006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4df0005990 tx=0x7f4df0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.117+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4df8049020 con 0x7f4e001f36d0 2024-04-12T19:51:09.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.322+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e001f7490 con 0x7f4e001f36d0 2024-04-12T19:51:09.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.325+0000 7f4df7400640 1 -- 172.21.15.32:0/2988875553 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f4df80aa120 con 0x7f4e001f36d0 2024-04-12T19:51:09.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:09.326 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:09.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 msgr2=0x7f4de006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 0x7f4de006ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4df0005990 tx=0x7f4df0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 msgr2=0x7f4e0015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4df802bf60 tx=0x7f4df8003a70 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 shutdown_connections 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4de006c920 0x7f4de006ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e001f36d0 0x7f4e0015ee10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e00160c20 0x7f4e001f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 --2- 172.21.15.32:0/2988875553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de40051c0 0x7f4e00152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 >> 172.21.15.32:0/2988875553 conn(0x7f4e0006b930 msgr2=0x7f4e0015b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.328+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 shutdown_connections 2024-04-12T19:51:09.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.329+0000 7f4e04a00640 1 -- 172.21.15.32:0/2988875553 wait complete. 2024-04-12T19:51:09.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:09.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.572+0000 7fcc0ca00640 1 Processor -- start 2024-04-12T19:51:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.572+0000 7fcc0ca00640 1 -- start start 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc0ca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc08147fc0 0x7fcc080b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc0ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc080aef00 0x7fcc081623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc0ca00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc080b5e00 con 0x7fcc08162930 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc0ca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcc081483a0 con 0x7fcc08147fc0 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc0ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080afaa0 con 0x7fcc080aef00 2024-04-12T19:51:09.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc07400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc08147fc0 0x7fcc080b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc07e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc08162930 0x7fcc080add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33278/0 (socket says 172.21.15.32:33278) 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.574+0000 7fcc07e00640 1 -- 172.21.15.32:0/1859086694 learned_addr learned my addr 172.21.15.32:0/1859086694 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.575+0000 7fcc06a00640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc080aef00 0x7fcc081623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.575+0000 7fcc07400640 1 -- 172.21.15.32:0/1859086694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc080aef00 msgr2=0x7fcc081623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.575+0000 7fcc07400640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc080aef00 0x7fcc081623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.575+0000 7fcc07400640 1 -- 172.21.15.32:0/1859086694 >> v1:172.21.15.32:6789/0 conn(0x7fcc08162930 legacy=0x7fcc080add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.575+0000 7fcc07400640 1 -- 172.21.15.32:0/1859086694 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc08162d50 con 0x7fcc08147fc0 2024-04-12T19:51:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.576+0000 7fcc07400640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08147fc0 0x7fcc080b36a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7b983e6ea561db server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.576+0000 7fcc06000640 1 -- 172.21.15.32:0/1859086694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8031780 con 0x7fcc08147fc0 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.576+0000 7fcc06000640 1 -- 172.21.15.32:0/1859086694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf8037070 con 0x7fcc08147fc0 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.576+0000 7fcc06000640 1 -- 172.21.15.32:0/1859086694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8031b40 con 0x7fcc08147fc0 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 -- 172.21.15.32:0/1859086694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08147fc0 msgr2=0x7fcc080b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08147fc0 0x7fcc080b36a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80097a0 tx=0x7fcbf802b820 comp rx=0 tx=0).stop 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 -- 172.21.15.32:0/1859086694 shutdown_connections 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc080aef00 0x7fcc081623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/1859086694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08147fc0 0x7fcc080b36a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 -- 172.21.15.32:0/1859086694 >> 172.21.15.32:0/1859086694 conn(0x7fcc080b23a0 msgr2=0x7fcc080b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 -- 172.21.15.32:0/1859086694 shutdown_connections 2024-04-12T19:51:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.577+0000 7fcc0ca00640 1 -- 172.21.15.32:0/1859086694 wait complete. 2024-04-12T19:51:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.579+0000 7fcc0ca00640 1 Processor -- start 2024-04-12T19:51:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.579+0000 7fcc0ca00640 1 -- start start 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc080aef00 0x7fcc08131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 0x7fcc08160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080b0060 con 0x7fcc08162930 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc080b6070 con 0x7fcc080aef00 2024-04-12T19:51:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc0ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc080b3be0 con 0x7fcc08147fc0 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43988/0 (socket says 172.21.15.32:43988) 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.580+0000 7fcc07e00640 1 -- 172.21.15.32:0/454543223 learned_addr learned my addr 172.21.15.32:0/454543223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc06a00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 0x7fcc08160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07400640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc080aef00 0x7fcc08131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 -- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 msgr2=0x7fcc08160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 0x7fcc08160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 -- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc080aef00 msgr2=0x7fcc08131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc080aef00 0x7fcc08131650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 -- 172.21.15.32:0/454543223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc08140cb0 con 0x7fcc08162930 2024-04-12T19:51:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc06a00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 0x7fcc08160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc07e00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000cbc0 tx=0x7fcc0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00017070 con 0x7fcc08162930 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0000ad40 con 0x7fcc08162930 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00012600 con 0x7fcc08162930 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.581+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc08140f90 con 0x7fcc08162930 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.582+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc08141550 con 0x7fcc08162930 2024-04-12T19:51:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.583+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0806ff20 con 0x7fcc08162930 2024-04-12T19:51:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.584+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc0001a070 con 0x7fcc08162930 2024-04-12T19:51:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.585+0000 7fcbff400640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.585+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fcc0008b850 con 0x7fcc08162930 2024-04-12T19:51:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.585+0000 7fcc07400640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:09.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.586+0000 7fcc07400640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8036000 tx=0x7fcbf803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.589+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc00059600 con 0x7fcc08162930 2024-04-12T19:51:09.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.793+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc080750a0 con 0x7fcc08162930 2024-04-12T19:51:09.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.794+0000 7fcbff400640 1 -- 172.21.15.32:0/454543223 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fcc0005d900 con 0x7fcc08162930 2024-04-12T19:51:09.794 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 msgr2=0x7fcbe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 0x7fcbe806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8036000 tx=0x7fcbf803d040 comp rx=0 tx=0).stop 2024-04-12T19:51:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 msgr2=0x7fcc081e7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000cbc0 tx=0x7fcc0000a430 comp rx=0 tx=0).stop 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 shutdown_connections 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcbe806c8f0 0x7fcbe806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08162930 0x7fcc081e7c30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc08147fc0 0x7fcc08160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 --2- 172.21.15.32:0/454543223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc080aef00 0x7fcc08131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.797+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 >> 172.21.15.32:0/454543223 conn(0x7fcc080b23a0 msgr2=0x7fcc080b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:09.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.798+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 shutdown_connections 2024-04-12T19:51:09.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:09.798+0000 7fcc0ca00640 1 -- 172.21.15.32:0/454543223 wait complete. 2024-04-12T19:51:09.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:09.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:51:09.971 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:51:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.036+0000 7f9e9e000640 1 Processor -- start 2024-04-12T19:51:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.036+0000 7f9e9e000640 1 -- start start 2024-04-12T19:51:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e9815ecc0 0x7f9e980d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 0x7f9e980b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e980b3720 0x7f9e980b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9e9815f0a0 con 0x7f9e980b3720 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980ba280 con 0x7f9e980d3270 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e9e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e980ba4d0 con 0x7f9e9815ecc0 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e97e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e980b3720 0x7f9e980b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e97e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e980b3720 0x7f9e980b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44004/0 (socket says 172.21.15.32:44004) 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.038+0000 7f9e97e00640 1 -- 172.21.15.32:0/2891630681 learned_addr learned my addr 172.21.15.32:0/2891630681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97400640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e9815ecc0 0x7f9e980d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e8ea00640 1 --2- 172.21.15.32:0/2891630681 >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 0x7f9e980b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97e00640 1 -- 172.21.15.32:0/2891630681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e9815ecc0 msgr2=0x7f9e980d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e9815ecc0 0x7f9e980d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97e00640 1 -- 172.21.15.32:0/2891630681 >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 msgr2=0x7f9e980b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2891630681 >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 0x7f9e980b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e97e00640 1 -- 172.21.15.32:0/2891630681 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e980ba7b0 con 0x7f9e980b3720 2024-04-12T19:51:10.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.039+0000 7f9e8ea00640 1 --2- 172.21.15.32:0/2891630681 >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 0x7f9e980b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.040+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e980b3720 0x7f9e980b9b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e980d59c0 tx=0x7f9e9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab95a9b52f596c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.040+0000 7f9e96a00640 1 -- 172.21.15.32:0/2891630681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9002ca80 con 0x7f9e980b3720 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.040+0000 7f9e96a00640 1 -- 172.21.15.32:0/2891630681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9002cc20 con 0x7f9e980b3720 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.040+0000 7f9e96a00640 1 -- 172.21.15.32:0/2891630681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900343f0 con 0x7f9e980b3720 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 -- 172.21.15.32:0/2891630681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e980b3720 msgr2=0x7f9e980b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e980b3720 0x7f9e980b9b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e980d59c0 tx=0x7f9e9002bee0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 -- 172.21.15.32:0/2891630681 shutdown_connections 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e980b3720 0x7f9e980b9b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2891630681 >> v2:172.21.15.169:3300/0 conn(0x7f9e980d3270 0x7f9e980b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2891630681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e9815ecc0 0x7f9e980d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 -- 172.21.15.32:0/2891630681 >> 172.21.15.32:0/2891630681 conn(0x7f9e98073480 msgr2=0x7f9e98073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 -- 172.21.15.32:0/2891630681 shutdown_connections 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.041+0000 7f9e9e000640 1 -- 172.21.15.32:0/2891630681 wait complete. 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.043+0000 7f9e9e000640 1 Processor -- start 2024-04-12T19:51:10.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.043+0000 7f9e9e000640 1 -- start start 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e980b3720 0x7f9e98161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e980d3270 0x7f9e98154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980b3c70 con 0x7f9e9815ecc0 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980b90d0 con 0x7f9e980b3720 2024-04-12T19:51:10.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.044+0000 7f9e9e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e980b3df0 con 0x7f9e980d3270 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44014/0 (socket says 172.21.15.32:44014) 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 -- 172.21.15.32:0/2783918759 learned_addr learned my addr 172.21.15.32:0/2783918759 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97400640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e980b3720 0x7f9e98161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e8ea00640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e980d3270 0x7f9e98154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 -- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e980d3270 msgr2=0x7f9e98154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e980d3270 0x7f9e98154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 -- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e980b3720 msgr2=0x7f9e98161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e980b3720 0x7f9e98161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.045+0000 7f9e97e00640 1 -- 172.21.15.32:0/2783918759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e98161aa0 con 0x7f9e9815ecc0 2024-04-12T19:51:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e97e00640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e90036040 tx=0x7f9e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90043020 con 0x7f9e9815ecc0 2024-04-12T19:51:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9003e450 con 0x7f9e9815ecc0 2024-04-12T19:51:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90009510 con 0x7f9e9815ecc0 2024-04-12T19:51:10.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e98161d80 con 0x7f9e9815ecc0 2024-04-12T19:51:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.046+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e98155e10 con 0x7f9e9815ecc0 2024-04-12T19:51:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.048+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e98068c80 con 0x7f9e9815ecc0 2024-04-12T19:51:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.048+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e90046020 con 0x7f9e9815ecc0 2024-04-12T19:51:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.049+0000 7f9e8fe00640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.049+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f9e900ae120 con 0x7f9e9815ecc0 2024-04-12T19:51:10.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.050+0000 7f9e97400640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.051+0000 7f9e97400640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 0x7f9e7406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006fd0 tx=0x7f9e88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.055+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e9004d070 con 0x7f9e9815ecc0 2024-04-12T19:51:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.268+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e980d03e0 con 0x7f9e9815ecc0 2024-04-12T19:51:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.269+0000 7f9e8fe00640 1 -- 172.21.15.32:0/2783918759 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f9e9007c520 con 0x7f9e9815ecc0 2024-04-12T19:51:10.269 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 msgr2=0x7f9e7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 0x7f9e7406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e88006fd0 tx=0x7f9e88008040 comp rx=0 tx=0).stop 2024-04-12T19:51:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 msgr2=0x7f9e9815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e90036040 tx=0x7f9e90004a40 comp rx=0 tx=0).stop 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 shutdown_connections 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e7406c920 0x7f9e7406ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815ecc0 0x7f9e9815ca90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e980d3270 0x7f9e98154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 --2- 172.21.15.32:0/2783918759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e980b3720 0x7f9e98161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 >> 172.21.15.32:0/2783918759 conn(0x7f9e98073480 msgr2=0x7f9e980b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 shutdown_connections 2024-04-12T19:51:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.272+0000 7f9e9e000640 1 -- 172.21.15.32:0/2783918759 wait complete. 2024-04-12T19:51:10.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.514+0000 7f42db200640 1 Processor -- start 2024-04-12T19:51:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.514+0000 7f42db200640 1 -- start start 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.515+0000 7f42db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42d40d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42db200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d40ba4d0 con 0x7f42d40d3270 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42db200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d40ba7b0 con 0x7f42d40b3720 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42d415f0a0 con 0x7f42d415ecc0 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42d3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42d40d3270 0x7f42d40b95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33294/0 (socket says 172.21.15.32:33294) 2024-04-12T19:51:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42d3400640 1 -- 172.21.15.32:0/2705876525 learned_addr learned my addr 172.21.15.32:0/2705876525 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42d40d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.516+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114317242 0 0) 0x7f42d40ba4d0 con 0x7f42d40d3270 2024-04-12T19:51:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b8003520 con 0x7f42d40d3270 2024-04-12T19:51:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2582573148 0 0) 0x7f42b8003520 con 0x7f42d40d3270 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 msgr2=0x7f42d40d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42d40d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 >> v1:172.21.15.169:6789/0 conn(0x7f42d40b3720 legacy=0x7f42d40cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d40b8df0 con 0x7f42d40d3270 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f42c0001670 con 0x7f42d40d3270 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 >> v1:172.21.15.32:6789/0 conn(0x7f42d40d3270 legacy=0x7f42d40b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80039e0 0x7f42b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b8004800 0x7f42b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42d415f0a0 con 0x7f42b80039e0 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42b800ba70 con 0x7f42b8004800 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.517+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42b800bc20 con 0x7f42d415ecc0 2024-04-12T19:51:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80039e0 0x7f42b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3400640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b8004800 0x7f42b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d9200640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 -- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 msgr2=0x7f42b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 -- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b8004800 msgr2=0x7f42b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b8004800 0x7f42b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d3e00640 1 -- 172.21.15.32:0/2705876525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b8003520 con 0x7f42b80039e0 2024-04-12T19:51:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.518+0000 7f42d9200640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42d415ecc0 0x7f42b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.519+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80039e0 0x7f42b8003dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009f90 tx=0x7f42b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50869ea82a55a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4039070 con 0x7f42b80039e0 2024-04-12T19:51:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c402cb90 con 0x7f42b80039e0 2024-04-12T19:51:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42d2a00640 1 -- 172.21.15.32:0/2705876525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4009c10 con 0x7f42b80039e0 2024-04-12T19:51:10.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42db200640 1 -- 172.21.15.32:0/2705876525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80039e0 msgr2=0x7f42b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42db200640 1 --2- 172.21.15.32:0/2705876525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80039e0 0x7f42b8003dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009f90 tx=0x7f42b8005830 comp rx=0 tx=0).stop 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.520+0000 7f42d9200640 1 -- 172.21.15.32:0/2705876525 reap_dead start 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.521+0000 7f42db200640 1 -- 172.21.15.32:0/2705876525 shutdown_connections 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.521+0000 7f42db200640 1 -- 172.21.15.32:0/2705876525 >> 172.21.15.32:0/2705876525 conn(0x7f42d4073480 msgr2=0x7f42d4073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.521+0000 7f42db200640 1 -- 172.21.15.32:0/2705876525 shutdown_connections 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.521+0000 7f42db200640 1 -- 172.21.15.32:0/2705876525 wait complete. 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.522+0000 7f42db200640 1 Processor -- start 2024-04-12T19:51:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 -- start start 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 0x7f42d40c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42d4161710 0x7f42d40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40b8aa0 con 0x7f42d40ce6d0 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40ba280 con 0x7f42d4161710 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.523+0000 7f42db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42d40b9d00 con 0x7f42b80039e0 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44040/0 (socket says 172.21.15.32:44040) 2024-04-12T19:51:10.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 -- 172.21.15.32:0/2536056507 learned_addr learned my addr 172.21.15.32:0/2536056507 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 0x7f42d40c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d3400640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42d4161710 0x7f42d40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 -- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 msgr2=0x7f42d40c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 0x7f42d40c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 -- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42d4161710 msgr2=0x7f42d40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42d4161710 0x7f42d40ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d9200640 1 -- 172.21.15.32:0/2536056507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d40c8510 con 0x7f42d40ce6d0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.524+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 0x7f42d40c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42d9200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42d40d24e0 tx=0x7f42cc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc018070 con 0x7f42d40ce6d0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42cc0035f0 con 0x7f42d40ce6d0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0039e0 con 0x7f42d40ce6d0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d40c87b0 con 0x7f42d40ce6d0 2024-04-12T19:51:10.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.525+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d40c8d70 con 0x7f42d40ce6d0 2024-04-12T19:51:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.526+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d4068c80 con 0x7f42d40ce6d0 2024-04-12T19:51:10.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.527+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42cc01b070 con 0x7f42d40ce6d0 2024-04-12T19:51:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.528+0000 7f42cbe00640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.528+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f42cc08b210 con 0x7f42d40ce6d0 2024-04-12T19:51:10.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.528+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.529+0000 7f42d3e00640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 0x7f42b406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42c4036000 tx=0x7f42c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.532+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42cc059580 con 0x7f42d40ce6d0 2024-04-12T19:51:10.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.741+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f42d40d3a90 con 0x7f42d40ce6d0 2024-04-12T19:51:10.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.742+0000 7f42cbe00640 1 -- 172.21.15.32:0/2536056507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f42cc058e80 con 0x7f42d40ce6d0 2024-04-12T19:51:10.742 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 msgr2=0x7f42b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 0x7f42b406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42c4036000 tx=0x7f42c402e000 comp rx=0 tx=0).stop 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 msgr2=0x7f42d41556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42d40d24e0 tx=0x7f42cc00d890 comp rx=0 tx=0).stop 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 shutdown_connections 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d40ce6d0 0x7f42d41556a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42d4161710 0x7f42d40ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 --2- 172.21.15.32:0/2536056507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b80039e0 0x7f42d40c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.744+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 >> 172.21.15.32:0/2536056507 conn(0x7f42d4073480 msgr2=0x7f42d40b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.745+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 shutdown_connections 2024-04-12T19:51:10.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.745+0000 7f42db200640 1 -- 172.21.15.32:0/2536056507 wait complete. 2024-04-12T19:51:10.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T19:51:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.969+0000 7f60a9200640 1 Processor -- start 2024-04-12T19:51:10.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.969+0000 7f60a9200640 1 -- start start 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60a415e870 0x7f60a409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 0x7f60a409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f60a409af80 0x7f60a41624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f60a415f930 con 0x7f60a415e870 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f60a415f000 con 0x7f60a409af80 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.970+0000 7f60a9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60a415f2e0 con 0x7f60a406ca40 2024-04-12T19:51:10.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60a415e870 0x7f60a409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60a415e870 0x7f60a409b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44048/0 (socket says 172.21.15.32:44048) 2024-04-12T19:51:10.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 -- 172.21.15.32:0/2861489114 learned_addr learned my addr 172.21.15.32:0/2861489114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f609be00640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 0x7f60a409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3e00640 1 --2- 172.21.15.32:0/2861489114 >> v2:172.21.15.169:3300/0 conn(0x7f60a409af80 0x7f60a41624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 -- 172.21.15.32:0/2861489114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 msgr2=0x7f60a409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 0x7f60a409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 -- 172.21.15.32:0/2861489114 >> v2:172.21.15.169:3300/0 conn(0x7f60a409af80 msgr2=0x7f60a41624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 --2- 172.21.15.32:0/2861489114 >> v2:172.21.15.169:3300/0 conn(0x7f60a409af80 0x7f60a41624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f60a3400640 1 -- 172.21.15.32:0/2861489114 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a415f5c0 con 0x7f60a415e870 2024-04-12T19:51:10.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.971+0000 7f609be00640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 0x7f60a409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.972+0000 7f60a3400640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a415e870 0x7f60a409b9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60a40b4ed0 tx=0x7f609402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c42ac63097d3eba server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.973+0000 7f60a2a00640 1 -- 172.21.15.32:0/2861489114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094035020 con 0x7f60a415e870 2024-04-12T19:51:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.973+0000 7f60a2a00640 1 -- 172.21.15.32:0/2861489114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609402d040 con 0x7f60a415e870 2024-04-12T19:51:10.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.973+0000 7f60a2a00640 1 -- 172.21.15.32:0/2861489114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094031a80 con 0x7f60a415e870 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 -- 172.21.15.32:0/2861489114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a415e870 msgr2=0x7f60a409b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a415e870 0x7f60a409b9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60a40b4ed0 tx=0x7f609402b900 comp rx=0 tx=0).stop 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 -- 172.21.15.32:0/2861489114 shutdown_connections 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 --2- 172.21.15.32:0/2861489114 >> v2:172.21.15.169:3300/0 conn(0x7f60a409af80 0x7f60a41624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a406ca40 0x7f60a409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 --2- 172.21.15.32:0/2861489114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a415e870 0x7f60a409b9b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 -- 172.21.15.32:0/2861489114 >> 172.21.15.32:0/2861489114 conn(0x7f60a40b40a0 msgr2=0x7f60a40b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 -- 172.21.15.32:0/2861489114 shutdown_connections 2024-04-12T19:51:10.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.974+0000 7f60a9200640 1 -- 172.21.15.32:0/2861489114 wait complete. 2024-04-12T19:51:10.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.975+0000 7f60a9200640 1 Processor -- start 2024-04-12T19:51:10.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.976+0000 7f60a9200640 1 -- start start 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a409af80 0x7f60a40d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a415e870 0x7f60a4155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f60a409bf90 con 0x7f60a406ca40 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60a415fdc0 con 0x7f60a415e870 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60a415ec50 con 0x7f60a409af80 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44050/0 (socket says 172.21.15.32:44050) 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 -- 172.21.15.32:0/1829366188 learned_addr learned my addr 172.21.15.32:0/1829366188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f609be00640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a409af80 0x7f60a40d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3e00640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a415e870 0x7f60a4155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 -- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a409af80 msgr2=0x7f60a40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a409af80 0x7f60a40d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 -- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a415e870 msgr2=0x7f60a4155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a415e870 0x7f60a4155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.977+0000 7f60a3400640 1 -- 172.21.15.32:0/1829366188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a40cf2c0 con 0x7f60a406ca40 2024-04-12T19:51:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f60a3400640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f609402b940 tx=0x7f60940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094040070 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6094039c30 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094035040 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a40d1550 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.978+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a40d1b10 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.980+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a40755c0 con 0x7f60a406ca40 2024-04-12T19:51:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.980+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f609404b060 con 0x7f60a406ca40 2024-04-12T19:51:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.981+0000 7f609b400640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 0x7f608006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.981+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f60940ad850 con 0x7f60a406ca40 2024-04-12T19:51:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.982+0000 7f609be00640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 0x7f608006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.982+0000 7f609be00640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 0x7f608006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60a415d100 tx=0x7f608c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:10.990+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f609407bc50 con 0x7f60a406ca40 2024-04-12T19:51:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.200+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60a415e090 con 0x7f60a406ca40 2024-04-12T19:51:11.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.201+0000 7f609b400640 1 -- 172.21.15.32:0/1829366188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f609407b550 con 0x7f60a406ca40 2024-04-12T19:51:11.201 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-12T19:51:11.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.203+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 msgr2=0x7f608006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.203+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 0x7f608006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60a415d100 tx=0x7f608c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:11.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 msgr2=0x7f60a415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f609402b940 tx=0x7f60940046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 shutdown_connections 2024-04-12T19:51:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f608006c8f0 0x7f608006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60a415e870 0x7f60a4155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60a409af80 0x7f60a40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 --2- 172.21.15.32:0/1829366188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60a406ca40 0x7f60a415d530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 >> 172.21.15.32:0/1829366188 conn(0x7f60a40b40a0 msgr2=0x7f60a4070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 shutdown_connections 2024-04-12T19:51:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.204+0000 7f60a9200640 1 -- 172.21.15.32:0/1829366188 wait complete. 2024-04-12T19:51:11.223 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-12T19:51:11.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:11.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-12T19:51:11.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:51:11.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:11.313 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-12T19:51:11.363 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:11.363 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:11.417 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:11.417 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:11.462 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:11.463 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:11.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:11.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.767+0000 7f25dbe00640 1 Processor -- start 2024-04-12T19:51:11.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.767+0000 7f25dbe00640 1 -- start start 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.768+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc15b480 0x7f25dc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.768+0000 7f25dbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25dc15b860 0x7f25dc0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.768+0000 7f25dbe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25dc15f2b0 con 0x7f25dc0cdde0 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.768+0000 7f25dbe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f25dc15f060 con 0x7f25dc15b860 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.768+0000 7f25dbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc15fec0 con 0x7f25dc15b480 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25d2000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25dc0cdde0 0x7f25dc06ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33306/0 (socket says 172.21.15.32:33306) 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25d2000640 1 -- 172.21.15.32:0/234545836 learned_addr learned my addr 172.21.15.32:0/234545836 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:11.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25db400640 1 --2- 172.21.15.32:0/234545836 >> v2:172.21.15.32:3300/0 conn(0x7f25dc15b860 0x7f25dc0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25daa00640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc15b480 0x7f25dc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25da000640 1 -- 172.21.15.32:0/234545836 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523867384 0 0) 0x7f25dc15f2b0 con 0x7f25dc0cdde0 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25da000640 1 -- 172.21.15.32:0/234545836 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25c0003520 con 0x7f25dc0cdde0 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25db400640 1 -- 172.21.15.32:0/234545836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc15b480 msgr2=0x7f25dc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25db400640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc15b480 0x7f25dc0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25db400640 1 -- 172.21.15.32:0/234545836 >> v1:172.21.15.32:6789/0 conn(0x7f25dc0cdde0 legacy=0x7f25dc06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.769+0000 7f25db400640 1 -- 172.21.15.32:0/234545836 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc15f590 con 0x7f25dc15b860 2024-04-12T19:51:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25db400640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b860 0x7f25dc0b3fc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25dc15f1e0 tx=0x7f25d402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8156a3dc38e741ce server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25da000640 1 -- 172.21.15.32:0/234545836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4039070 con 0x7f25dc15b860 2024-04-12T19:51:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25da000640 1 -- 172.21.15.32:0/234545836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d402cc40 con 0x7f25dc15b860 2024-04-12T19:51:11.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25da000640 1 -- 172.21.15.32:0/234545836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4034440 con 0x7f25dc15b860 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25dbe00640 1 -- 172.21.15.32:0/234545836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b860 msgr2=0x7f25dc0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.771+0000 7f25dbe00640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b860 0x7f25dc0b3fc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25dc15f1e0 tx=0x7f25d402be90 comp rx=0 tx=0).stop 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 -- 172.21.15.32:0/234545836 shutdown_connections 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b860 0x7f25dc0b3fc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 --2- 172.21.15.32:0/234545836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc15b480 0x7f25dc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 -- 172.21.15.32:0/234545836 >> 172.21.15.32:0/234545836 conn(0x7f25dc06b930 msgr2=0x7f25dc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:11.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 -- 172.21.15.32:0/234545836 shutdown_connections 2024-04-12T19:51:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.772+0000 7f25dbe00640 1 -- 172.21.15.32:0/234545836 wait complete. 2024-04-12T19:51:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.773+0000 7f25dbe00640 1 Processor -- start 2024-04-12T19:51:11.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.774+0000 7f25dbe00640 1 -- start start 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.774+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc0cdde0 0x7f25dc0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.774+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.774+0000 7f25dbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 0x7f25dc1f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25dbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc0cd550 con 0x7f25dc15b480 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25dbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc15fb50 con 0x7f25dc15b860 2024-04-12T19:51:11.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25dbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc06f4f0 con 0x7f25dc0cdde0 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44064/0 (socket says 172.21.15.32:44064) 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 -- 172.21.15.32:0/959176784 learned_addr learned my addr 172.21.15.32:0/959176784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25daa00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc0cdde0 0x7f25dc0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25db400640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 0x7f25dc1f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 -- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc0cdde0 msgr2=0x7f25dc0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc0cdde0 0x7f25dc0b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 -- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 msgr2=0x7f25dc1f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 0x7f25dc1f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25d2000640 1 -- 172.21.15.32:0/959176784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc0c9710 con 0x7f25dc15b480 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.775+0000 7f25db400640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 0x7f25dc1f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25d2000640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c430 tx=0x7f25c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25dc15b480 2024-04-12T19:51:11.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c4005ce0 con 0x7f25dc15b480 2024-04-12T19:51:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40072f0 con 0x7f25dc15b480 2024-04-12T19:51:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc0c99f0 con 0x7f25dc15b480 2024-04-12T19:51:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.776+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc0c7b30 con 0x7f25dc15b480 2024-04-12T19:51:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.777+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc0c7d30 con 0x7f25dc15b480 2024-04-12T19:51:11.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.778+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25c401a070 con 0x7f25dc15b480 2024-04-12T19:51:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.779+0000 7f25d3400640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.779+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f25c408b060 con 0x7f25dc15b480 2024-04-12T19:51:11.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.779+0000 7f25daa00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:11.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.780+0000 7f25daa00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009ad0 tx=0x7f25cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:11.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.783+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25dc0c7d30 con 0x7f25dc15b480 2024-04-12T19:51:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.992+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25dc06e8a0 con 0x7f25dc15b480 2024-04-12T19:51:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.995+0000 7f25d3400640 1 -- 172.21.15.32:0/959176784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f25c4059460 con 0x7f25dc15b480 2024-04-12T19:51:11.995 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:11.996 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.997+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 msgr2=0x7f25b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.997+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 0x7f25b806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009ad0 tx=0x7f25cc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.997+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 msgr2=0x7f25dc1f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.997+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25c400c430 tx=0x7f25c400c8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 shutdown_connections 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f25b806c8f0 0x7f25b806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc15b860 0x7f25dc1f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25dc15b480 0x7f25dc1f1ee0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 --2- 172.21.15.32:0/959176784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25dc0cdde0 0x7f25dc0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 >> 172.21.15.32:0/959176784 conn(0x7f25dc06b930 msgr2=0x7f25dc15bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 shutdown_connections 2024-04-12T19:51:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:11.998+0000 7f25dbe00640 1 -- 172.21.15.32:0/959176784 wait complete. 2024-04-12T19:51:12.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:12.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.239+0000 7f262f000640 1 Processor -- start 2024-04-12T19:51:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.240+0000 7f262f000640 1 -- start start 2024-04-12T19:51:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.241+0000 7f262f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.241+0000 7f262f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26280cfd20 0x7f26280b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.241+0000 7f262f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280d2bb0 con 0x7f26280d26b0 2024-04-12T19:51:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.241+0000 7f262f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f26280d2e90 con 0x7f26280cfd20 2024-04-12T19:51:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.241+0000 7f262f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26280d3170 con 0x7f2628161f80 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26280cfd20 0x7f26280b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262e600640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f26280d26b0 0x7f26280d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:42438/0 (socket says 172.21.15.32:42438) 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26280cfd20 0x7f26280b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:53336/0 (socket says 172.21.15.32:53336) 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262d200640 1 -- 172.21.15.32:0/92943726 learned_addr learned my addr 172.21.15.32:0/92943726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.242+0000 7f262dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:52464/0 (socket says 172.21.15.32:52464) 2024-04-12T19:51:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f262d200640 1 -- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 msgr2=0x7f26280cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f262d200640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f262d200640 1 -- 172.21.15.32:0/92943726 >> v1:172.21.15.169:6789/0 conn(0x7f26280d26b0 legacy=0x7f26280d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f262d200640 1 -- 172.21.15.32:0/92943726 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280d3450 con 0x7f26280cfd20 2024-04-12T19:51:12.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f262dc00640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:12.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.243+0000 7f2627e00640 1 -- 172.21.15.32:0/92943726 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2412438771 0 0) 0x7f26280d2bb0 con 0x7f26280d26b0 2024-04-12T19:51:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.245+0000 7f262d200640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26280b7940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a0550b95e0e477 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.245+0000 7f2627e00640 1 -- 172.21.15.32:0/92943726 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614034070 con 0x7f26280cfd20 2024-04-12T19:51:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.245+0000 7f2627e00640 1 -- 172.21.15.32:0/92943726 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614030860 con 0x7f26280cfd20 2024-04-12T19:51:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.245+0000 7f2627e00640 1 -- 172.21.15.32:0/92943726 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614030c30 con 0x7f26280cfd20 2024-04-12T19:51:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 -- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 msgr2=0x7f26280b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26280b7940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f261402b9a0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 -- 172.21.15.32:0/92943726 shutdown_connections 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26280b7940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 --2- 172.21.15.32:0/92943726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26280cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 -- 172.21.15.32:0/92943726 >> 172.21.15.32:0/92943726 conn(0x7f262806ce40 msgr2=0x7f262806d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 -- 172.21.15.32:0/92943726 shutdown_connections 2024-04-12T19:51:12.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.246+0000 7f262f000640 1 -- 172.21.15.32:0/92943726 wait complete. 2024-04-12T19:51:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.248+0000 7f262f000640 1 Processor -- start 2024-04-12T19:51:12.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.248+0000 7f262f000640 1 -- start start 2024-04-12T19:51:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26281f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26281f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26280d1e10 con 0x7f26280d26b0 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26280d1c90 con 0x7f26280cfd20 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.249+0000 7f262f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26280b7e80 con 0x7f2628161f80 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44076/0 (socket says 172.21.15.32:44076) 2024-04-12T19:51:12.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 -- 172.21.15.32:0/2640951741 learned_addr learned my addr 172.21.15.32:0/2640951741 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262dc00640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26281f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262e600640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26281f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 -- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 msgr2=0x7f26281f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26281f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 -- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 msgr2=0x7f26281f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26281f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 -- 172.21.15.32:0/2640951741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26281f94e0 con 0x7f26280d26b0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.250+0000 7f262d200640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f2614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.251+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614048070 con 0x7f26280d26b0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.251+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614030660 con 0x7f26280d26b0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.251+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614034040 con 0x7f26280d26b0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.251+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2628155be0 con 0x7f26280d26b0 2024-04-12T19:51:12.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.251+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281561a0 con 0x7f26280d26b0 2024-04-12T19:51:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.252+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26280755c0 con 0x7f26280d26b0 2024-04-12T19:51:12.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.253+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2614037070 con 0x7f26280d26b0 2024-04-12T19:51:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.254+0000 7f2625600640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.254+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f26140ad8d0 con 0x7f26280d26b0 2024-04-12T19:51:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.254+0000 7f262dc00640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.255+0000 7f262dc00640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 0x7f260806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2628151ca0 tx=0x7f261c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.258+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26281f94e0 con 0x7f26280d26b0 2024-04-12T19:51:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.463+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26280d2490 con 0x7f26280d26b0 2024-04-12T19:51:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.464+0000 7f2625600640 1 -- 172.21.15.32:0/2640951741 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f261407be90 con 0x7f26280d26b0 2024-04-12T19:51:12.464 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 msgr2=0x7f260806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 0x7f260806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2628151ca0 tx=0x7f261c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 msgr2=0x7f2628152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2614009870 tx=0x7f2614004730 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 shutdown_connections 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2628161f80 0x7f26281f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26280d26b0 0x7f2628152730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 --2- 172.21.15.32:0/2640951741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26280cfd20 0x7f26281f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 >> 172.21.15.32:0/2640951741 conn(0x7f262806ce40 msgr2=0x7f262806d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 shutdown_connections 2024-04-12T19:51:12.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.467+0000 7f262f000640 1 -- 172.21.15.32:0/2640951741 wait complete. 2024-04-12T19:51:12.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.720+0000 7f9aebe00640 1 Processor -- start 2024-04-12T19:51:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.720+0000 7f9aebe00640 1 -- start start 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aebe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9aec0cebb0 0x7f9aec167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aebe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec0cd2d0 con 0x7f9aec1461c0 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aebe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec0cd5b0 con 0x7f9aec0725f0 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aebe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9aec0cf1d0 con 0x7f9aec0cebb0 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aeaa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9aec1461c0 0x7f9aec071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33316/0 (socket says 172.21.15.32:33316) 2024-04-12T19:51:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aeaa00640 1 -- 172.21.15.32:0/1963976280 learned_addr learned my addr 172.21.15.32:0/1963976280 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.722+0000 7f9aeb400640 1 --2- 172.21.15.32:0/1963976280 >> v2:172.21.15.169:3300/0 conn(0x7f9aec0cebb0 0x7f9aec167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697741744 0 0) 0x7f9aec0cd2d0 con 0x7f9aec1461c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ad0003520 con 0x7f9aec1461c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3507443435 0 0) 0x7f9ad0003520 con 0x7f9aec1461c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 >> v2:172.21.15.169:3300/0 conn(0x7f9aec0cebb0 msgr2=0x7f9aec167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 --2- 172.21.15.32:0/1963976280 >> v2:172.21.15.169:3300/0 conn(0x7f9aec0cebb0 0x7f9aec167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 >> v1:172.21.15.169:6789/0 conn(0x7f9aec0725f0 legacy=0x7f9aec0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0cd890 con 0x7f9aec1461c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f9adc001670 con 0x7f9aec1461c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 >> v1:172.21.15.32:6789/0 conn(0x7f9aec1461c0 legacy=0x7f9aec071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00051c0 0x7f9ad000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad000c090 con 0x7f9ad0004800 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad000c240 con 0x7f9ad00051c0 2024-04-12T19:51:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad000c440 con 0x7f9ad0003a00 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.723+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9aeaa00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9aeb400640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00051c0 0x7f9ad000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9ae3e00640 1 -- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0003a00 msgr2=0x7f9ad0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0003a00 0x7f9ad0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9ae3e00640 1 -- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00051c0 msgr2=0x7f9ad000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00051c0 0x7f9ad000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9ae3e00640 1 -- 172.21.15.32:0/1963976280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad0003520 con 0x7f9ad0004800 2024-04-12T19:51:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.724+0000 7f9aeb400640 1 -- 172.21.15.32:0/1963976280 reap_dead start 2024-04-12T19:51:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.725+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 0x7f9ad0004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4001750 tx=0x7f9ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e94c312b7076b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.725+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4001370 con 0x7f9ad0004800 2024-04-12T19:51:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.725+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad40052d0 con 0x7f9ad0004800 2024-04-12T19:51:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.725+0000 7f9aea000640 1 -- 172.21.15.32:0/1963976280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003830 con 0x7f9ad0004800 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 -- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 msgr2=0x7f9ad0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 0x7f9ad0004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4001750 tx=0x7f9ad0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 -- 172.21.15.32:0/1963976280 shutdown_connections 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 --2- 172.21.15.32:0/1963976280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ad0004800 0x7f9ad0004c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 -- 172.21.15.32:0/1963976280 >> 172.21.15.32:0/1963976280 conn(0x7f9aec06b930 msgr2=0x7f9aec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 -- 172.21.15.32:0/1963976280 shutdown_connections 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.726+0000 7f9aebe00640 1 -- 172.21.15.32:0/1963976280 wait complete. 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.728+0000 7f9aebe00640 1 Processor -- start 2024-04-12T19:51:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.728+0000 7f9aebe00640 1 -- start start 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad0004800 0x7f9aec1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aec0aaa80 0x7f9aec0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0ce4a0 con 0x7f9aec0a94f0 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0cd080 con 0x7f9ad0004800 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9aebe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec0ce620 con 0x7f9aec0aaa80 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9ae3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9ae3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44094/0 (socket says 172.21.15.32:44094) 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.729+0000 7f9ae3e00640 1 -- 172.21.15.32:0/2313263965 learned_addr learned my addr 172.21.15.32:0/2313263965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 -- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aec0aaa80 msgr2=0x7f9aec0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aec0aaa80 0x7f9aec0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 -- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad0004800 msgr2=0x7f9aec1f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9aeaa00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad0004800 0x7f9aec1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad0004800 0x7f9aec1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 -- 172.21.15.32:0/2313263965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0a9a60 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.730+0000 7f9ae3e00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4005040 tx=0x7f9ad4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.731+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4041020 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.731+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4005480 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.731+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4007a30 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.731+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec0a9d40 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.731+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec0aa300 con 0x7f9aec0a94f0 2024-04-12T19:51:12.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.732+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec0755c0 con 0x7f9aec0a94f0 2024-04-12T19:51:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.733+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ad4044020 con 0x7f9aec0a94f0 2024-04-12T19:51:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.734+0000 7f9ae3400640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 0x7f9ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.734+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f9ad40ad5e0 con 0x7f9aec0a94f0 2024-04-12T19:51:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.734+0000 7f9aeaa00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 0x7f9ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.735+0000 7f9aeaa00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 0x7f9ac806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9adc001eb0 tx=0x7f9adc000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.740+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ad404b030 con 0x7f9aec0a94f0 2024-04-12T19:51:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.942+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9aec072e40 con 0x7f9aec0a94f0 2024-04-12T19:51:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.943+0000 7f9ae3400640 1 -- 172.21.15.32:0/2313263965 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f9ad40303c0 con 0x7f9aec0a94f0 2024-04-12T19:51:12.943 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 msgr2=0x7f9ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 0x7f9ac806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9adc001eb0 tx=0x7f9adc000ed0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 msgr2=0x7f9aec1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4005040 tx=0x7f9ad4001120 comp rx=0 tx=0).stop 2024-04-12T19:51:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 shutdown_connections 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ac806c8f0 0x7f9ac806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aec0aaa80 0x7f9aec0aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aec0a94f0 0x7f9aec1f8350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 --2- 172.21.15.32:0/2313263965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad0004800 0x7f9aec1f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 >> 172.21.15.32:0/2313263965 conn(0x7f9aec06b930 msgr2=0x7f9aec06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 shutdown_connections 2024-04-12T19:51:12.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:12.946+0000 7f9aebe00640 1 -- 172.21.15.32:0/2313263965 wait complete. 2024-04-12T19:51:12.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.189+0000 7fb9afe00640 1 Processor -- start 2024-04-12T19:51:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.189+0000 7fb9afe00640 1 -- start start 2024-04-12T19:51:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9a815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9afe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9a8072150 0x7fb9a80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9afe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a815d520 con 0x7fb9a815a440 2024-04-12T19:51:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9afe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a8072b00 con 0x7fb9a8072150 2024-04-12T19:51:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9afe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a806e280 con 0x7fb9a806fc80 2024-04-12T19:51:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9ad400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9a815a440 0x7fb9a80cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33322/0 (socket says 172.21.15.32:33322) 2024-04-12T19:51:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.191+0000 7fb9ad400640 1 -- 172.21.15.32:0/395150136 learned_addr learned my addr 172.21.15.32:0/395150136 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9ade00640 1 --2- 172.21.15.32:0/395150136 >> v2:172.21.15.169:3300/0 conn(0x7fb9a8072150 0x7fb9a80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9aca00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9a815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709941639 0 0) 0x7fb9a815d520 con 0x7fb9a815a440 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb990003520 con 0x7fb9a815a440 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 452044151 0 0) 0x7fb990003520 con 0x7fb9a815a440 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 msgr2=0x7fb9a815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9a815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 >> v2:172.21.15.169:3300/0 conn(0x7fb9a8072150 msgr2=0x7fb9a80725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 --2- 172.21.15.32:0/395150136 >> v2:172.21.15.169:3300/0 conn(0x7fb9a8072150 0x7fb9a80725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a815d800 con 0x7fb9a815a440 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fb99c001690 con 0x7fb9a815a440 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 >> v1:172.21.15.32:6789/0 conn(0x7fb9a815a440 legacy=0x7fb9a80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.192+0000 7fb9a7e00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9a7e00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb990003f30 0x7fb990004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9a7e00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb990004d40 0x7fb9900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8072b00 con 0x7fb990004d40 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a806e280 con 0x7fb990003f30 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb99000b9c0 con 0x7fb9a806fc80 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb990004d40 0x7fb9900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ad400640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9aca00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb990003f30 0x7fb990004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 -- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 msgr2=0x7fb9900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a806fc80 0x7fb9900039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 -- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb990003f30 msgr2=0x7fb990004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb990003f30 0x7fb990004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.193+0000 7fb9ade00640 1 -- 172.21.15.32:0/395150136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990003520 con 0x7fb990004d40 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.194+0000 7fb9ade00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb990004d40 0x7fb9900051c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812fc86ccb61c4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0039070 con 0x7fb990004d40 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a002cc40 con 0x7fb990004d40 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9a7e00640 1 -- 172.21.15.32:0/395150136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0009c10 con 0x7fb990004d40 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9afe00640 1 -- 172.21.15.32:0/395150136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb990004d40 msgr2=0x7fb9900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9afe00640 1 --2- 172.21.15.32:0/395150136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb990004d40 0x7fb9900051c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009fc0 tx=0x7fb990005830 comp rx=0 tx=0).stop 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.195+0000 7fb9ade00640 1 -- 172.21.15.32:0/395150136 reap_dead start 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.196+0000 7fb9afe00640 1 -- 172.21.15.32:0/395150136 shutdown_connections 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.196+0000 7fb9afe00640 1 -- 172.21.15.32:0/395150136 >> 172.21.15.32:0/395150136 conn(0x7fb9a80b2460 msgr2=0x7fb9a80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.196+0000 7fb9afe00640 1 -- 172.21.15.32:0/395150136 shutdown_connections 2024-04-12T19:51:13.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.196+0000 7fb9afe00640 1 -- 172.21.15.32:0/395150136 wait complete. 2024-04-12T19:51:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.198+0000 7fb9afe00640 1 Processor -- start 2024-04-12T19:51:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.198+0000 7fb9afe00640 1 -- start start 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb990004d40 0x7fb9a81f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a815c9f0 0x7fb9a80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a815a820 con 0x7fb9a81f5970 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cd7f0 con 0x7fb9a815c9f0 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9afe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a815ddc0 con 0x7fb990004d40 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44116/0 (socket says 172.21.15.32:44116) 2024-04-12T19:51:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 -- 172.21.15.32:0/583338119 learned_addr learned my addr 172.21.15.32:0/583338119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9ad400640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb990004d40 0x7fb9a81f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9ade00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a815c9f0 0x7fb9a80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 -- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb990004d40 msgr2=0x7fb9a81f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb990004d40 0x7fb9a81f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 -- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a815c9f0 msgr2=0x7fb9a80b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a815c9f0 0x7fb9a80b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.199+0000 7fb9aca00640 1 -- 172.21.15.32:0/583338119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a81f5770 con 0x7fb9a81f5970 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9aca00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb994010860 tx=0x7fb994010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99400f040 con 0x7fb9a81f5970 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99400eb60 con 0x7fb9a81f5970 2024-04-12T19:51:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a80c5d30 con 0x7fb9a81f5970 2024-04-12T19:51:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99401c400 con 0x7fb9a81f5970 2024-04-12T19:51:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.200+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80c5a50 con 0x7fb9a81f5970 2024-04-12T19:51:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.202+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a8075860 con 0x7fb9a81f5970 2024-04-12T19:51:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.203+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb99401c5a0 con 0x7fb9a81f5970 2024-04-12T19:51:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.204+0000 7fb9a5600640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 0x7fb98806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.204+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fb99408c1c0 con 0x7fb9a81f5970 2024-04-12T19:51:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.204+0000 7fb9ad400640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 0x7fb98806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.205+0000 7fb9ad400640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 0x7fb98806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0011e0 tx=0x7fb99c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.208+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9a8075860 con 0x7fb9a81f5970 2024-04-12T19:51:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.416+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9a815e7b0 con 0x7fb9a81f5970 2024-04-12T19:51:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.417+0000 7fb9a5600640 1 -- 172.21.15.32:0/583338119 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb9a81f5770 con 0x7fb9a81f5970 2024-04-12T19:51:13.417 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 msgr2=0x7fb98806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 0x7fb98806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0011e0 tx=0x7fb99c009040 comp rx=0 tx=0).stop 2024-04-12T19:51:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 msgr2=0x7fb9a81f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb994010860 tx=0x7fb994010d20 comp rx=0 tx=0).stop 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 shutdown_connections 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb98806c920 0x7fb98806ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a815c9f0 0x7fb9a80b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9a81f5970 0x7fb9a81f7550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 --2- 172.21.15.32:0/583338119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb990004d40 0x7fb9a81f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 >> 172.21.15.32:0/583338119 conn(0x7fb9a80b2460 msgr2=0x7fb9a806e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 shutdown_connections 2024-04-12T19:51:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.420+0000 7fb9afe00640 1 -- 172.21.15.32:0/583338119 wait complete. 2024-04-12T19:51:13.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T19:51:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.667+0000 7fda35c00640 1 Processor -- start 2024-04-12T19:51:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.667+0000 7fda35c00640 1 -- start start 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.668+0000 7fda35c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda3015eab0 0x7fda30162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda35c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda30162b70 0x7fda300d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda35c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda300b78f0 con 0x7fda300d1a90 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda35c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fda300d2e50 con 0x7fda30162b70 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda35c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fda30163010 con 0x7fda3015eab0 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda2f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda30162b70 0x7fda300d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda2f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda30162b70 0x7fda300d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44120/0 (socket says 172.21.15.32:44120) 2024-04-12T19:51:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda2f400640 1 -- 172.21.15.32:0/2220319244 learned_addr learned my addr 172.21.15.32:0/2220319244 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.669+0000 7fda2fe00640 1 --2- 172.21.15.32:0/2220319244 >> v2:172.21.15.169:3300/0 conn(0x7fda3015eab0 0x7fda30162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.670+0000 7fda2f400640 1 -- 172.21.15.32:0/2220319244 >> v2:172.21.15.169:3300/0 conn(0x7fda3015eab0 msgr2=0x7fda30162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.670+0000 7fda2f400640 1 --2- 172.21.15.32:0/2220319244 >> v2:172.21.15.169:3300/0 conn(0x7fda3015eab0 0x7fda30162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.670+0000 7fda2f400640 1 -- 172.21.15.32:0/2220319244 >> v1:172.21.15.169:6789/0 conn(0x7fda300d1a90 legacy=0x7fda300b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.670+0000 7fda2f400640 1 -- 172.21.15.32:0/2220319244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301632f0 con 0x7fda30162b70 2024-04-12T19:51:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.671+0000 7fda2f400640 1 --2- 172.21.15.32:0/2220319244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda30162b70 0x7fda300d1550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda3008d2c0 tx=0x7fda1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d21a376a27b78a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.671+0000 7fda2ea00640 1 -- 172.21.15.32:0/2220319244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c035020 con 0x7fda30162b70 2024-04-12T19:51:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.671+0000 7fda2ea00640 1 -- 172.21.15.32:0/2220319244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c031870 con 0x7fda30162b70 2024-04-12T19:51:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.671+0000 7fda2ea00640 1 -- 172.21.15.32:0/2220319244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c031c40 con 0x7fda30162b70 2024-04-12T19:51:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 -- 172.21.15.32:0/2220319244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda30162b70 msgr2=0x7fda300d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 --2- 172.21.15.32:0/2220319244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda30162b70 0x7fda300d1550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda3008d2c0 tx=0x7fda1c02b910 comp rx=0 tx=0).stop 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 -- 172.21.15.32:0/2220319244 shutdown_connections 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 --2- 172.21.15.32:0/2220319244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda30162b70 0x7fda300d1550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 --2- 172.21.15.32:0/2220319244 >> v2:172.21.15.169:3300/0 conn(0x7fda3015eab0 0x7fda30162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 -- 172.21.15.32:0/2220319244 >> 172.21.15.32:0/2220319244 conn(0x7fda30073240 msgr2=0x7fda30073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 -- 172.21.15.32:0/2220319244 shutdown_connections 2024-04-12T19:51:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.672+0000 7fda35c00640 1 -- 172.21.15.32:0/2220319244 wait complete. 2024-04-12T19:51:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.674+0000 7fda35c00640 1 Processor -- start 2024-04-12T19:51:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.674+0000 7fda35c00640 1 -- start start 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda300d1a90 0x7fda3014e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda30162b70 0x7fda300cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda300ba590 con 0x7fda3015eab0 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda3015ee90 con 0x7fda30162b70 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.675+0000 7fda35c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda300ba710 con 0x7fda300d1a90 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44130/0 (socket says 172.21.15.32:44130) 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 -- 172.21.15.32:0/2539183333 learned_addr learned my addr 172.21.15.32:0/2539183333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2fe00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda300d1a90 0x7fda3014e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda35200640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda30162b70 0x7fda300cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 -- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda300d1a90 msgr2=0x7fda3014e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda300d1a90 0x7fda3014e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 -- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda30162b70 msgr2=0x7fda300cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda30162b70 0x7fda300cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.676+0000 7fda2f400640 1 -- 172.21.15.32:0/2539183333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda300cccf0 con 0x7fda3015eab0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda2f400640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda1c02b950 tx=0x7fda1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c040070 con 0x7fda3015eab0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1c030d00 con 0x7fda3015eab0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda300ccfd0 con 0x7fda3015eab0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c035040 con 0x7fda3015eab0 2024-04-12T19:51:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.677+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda300cd590 con 0x7fda3015eab0 2024-04-12T19:51:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.679+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda300755c0 con 0x7fda3015eab0 2024-04-12T19:51:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.680+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda1c04b060 con 0x7fda3015eab0 2024-04-12T19:51:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.681+0000 7fda27e00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.681+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fda1c0ad9a0 con 0x7fda3015eab0 2024-04-12T19:51:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.681+0000 7fda2fe00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.682+0000 7fda2fe00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 0x7fda1006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.685+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda300b78f0 con 0x7fda3015eab0 2024-04-12T19:51:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.901+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda300d2460 con 0x7fda3015eab0 2024-04-12T19:51:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.901+0000 7fda27e00640 1 -- 172.21.15.32:0/2539183333 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fda1c07bdb0 con 0x7fda3015eab0 2024-04-12T19:51:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.904+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 msgr2=0x7fda1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.904+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 0x7fda1006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.904+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 msgr2=0x7fda301f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.904+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda1c02b950 tx=0x7fda1c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 shutdown_connections 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fda1006c8f0 0x7fda1006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda30162b70 0x7fda300cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3015eab0 0x7fda301f4db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 --2- 172.21.15.32:0/2539183333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda300d1a90 0x7fda3014e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:13.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 >> 172.21.15.32:0/2539183333 conn(0x7fda30073240 msgr2=0x7fda300c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 shutdown_connections 2024-04-12T19:51:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:13.905+0000 7fda35c00640 1 -- 172.21.15.32:0/2539183333 wait complete. 2024-04-12T19:51:13.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-12T19:51:13.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:13.927 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-12T19:51:14.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:51:14.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:14.038 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-12T19:51:14.089 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:14.090 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:14.148 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:14.148 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:14.204 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:14.205 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:14.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:14.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.485+0000 7f7adde00640 1 Processor -- start 2024-04-12T19:51:14.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.485+0000 7f7adde00640 1 -- start start 2024-04-12T19:51:14.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7adde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ad806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7adde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ad815d2f0 0x7f7ad80723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7adde00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad815dbb0 con 0x7f7ad8071f50 2024-04-12T19:51:14.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7adde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad806f4f0 con 0x7f7ad815d2f0 2024-04-12T19:51:14.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7adde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad80728f0 con 0x7f7ad80cee00 2024-04-12T19:51:14.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7ad7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7ad8071f50 0x7f7ad815cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54712/0 (socket says 172.21.15.32:54712) 2024-04-12T19:51:14.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.487+0000 7f7ad7e00640 1 -- 172.21.15.32:0/1490559368 learned_addr learned my addr 172.21.15.32:0/1490559368 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:14.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7adca00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ad806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7add400640 1 --2- 172.21.15.32:0/1490559368 >> v2:172.21.15.169:3300/0 conn(0x7f7ad815d2f0 0x7f7ad80723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210828234 0 0) 0x7f7ad815dbb0 con 0x7f7ad8071f50 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac0003520 con 0x7f7ad8071f50 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2646043602 0 0) 0x7f7ac0003520 con 0x7f7ad8071f50 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 msgr2=0x7f7ad806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ad806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 >> v2:172.21.15.169:3300/0 conn(0x7f7ad815d2f0 msgr2=0x7f7ad80723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 --2- 172.21.15.32:0/1490559368 >> v2:172.21.15.169:3300/0 conn(0x7f7ad815d2f0 0x7f7ad80723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad815d8d0 con 0x7f7ad8071f50 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f7ac4001670 con 0x7f7ad8071f50 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 >> v1:172.21.15.32:6789/0 conn(0x7f7ad8071f50 legacy=0x7f7ad815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ac00039b0 0x7f7ac0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ac0004800 0x7f7ac0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac000b9c0 con 0x7f7ac00039b0 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.488+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac000bb40 con 0x7f7ac0004800 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac000bcc0 con 0x7f7ad80cee00 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ac00039b0 0x7f7ac0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7ad7e00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ac0004800 0x7f7ac0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7add400640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 -- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 msgr2=0x7f7ac00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ad80cee00 0x7f7ac00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 -- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ac0004800 msgr2=0x7f7ac0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ac0004800 0x7f7ac0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.489+0000 7f7adca00640 1 -- 172.21.15.32:0/1490559368 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac0003520 con 0x7f7ac00039b0 2024-04-12T19:51:14.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.490+0000 7f7adca00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ac00039b0 0x7f7ac0003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009ac0 tx=0x7f7ac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823561267426d816 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc039070 con 0x7f7ac00039b0 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc02c6b0 con 0x7f7ac00039b0 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7ad7400640 1 -- 172.21.15.32:0/1490559368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc009690 con 0x7f7ac00039b0 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 -- 172.21.15.32:0/1490559368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ac00039b0 msgr2=0x7f7ac0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 --2- 172.21.15.32:0/1490559368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ac00039b0 0x7f7ac0003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009ac0 tx=0x7f7ac0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7add400640 1 -- 172.21.15.32:0/1490559368 reap_dead start 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 -- 172.21.15.32:0/1490559368 shutdown_connections 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 -- 172.21.15.32:0/1490559368 >> 172.21.15.32:0/1490559368 conn(0x7f7ad806b930 msgr2=0x7f7ad806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 -- 172.21.15.32:0/1490559368 shutdown_connections 2024-04-12T19:51:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.491+0000 7f7adde00640 1 -- 172.21.15.32:0/1490559368 wait complete. 2024-04-12T19:51:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.493+0000 7f7adde00640 1 Processor -- start 2024-04-12T19:51:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.493+0000 7f7adde00640 1 -- start start 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ac00039b0 0x7f7ad8141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 0x7f7ad8153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad806e3a0 con 0x7f7ad8151a50 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad806e0c0 con 0x7f7ad81534e0 2024-04-12T19:51:14.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.494+0000 7f7adde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad806e8c0 con 0x7f7ac00039b0 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58558/0 (socket says 172.21.15.32:58558) 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 -- 172.21.15.32:0/4240921946 learned_addr learned my addr 172.21.15.32:0/4240921946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7ad7e00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 0x7f7ad8153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7adca00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ac00039b0 0x7f7ad8141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 -- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ac00039b0 msgr2=0x7f7ad8141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ac00039b0 0x7f7ad8141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 -- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 msgr2=0x7f7ad8153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 0x7f7ad8153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7add400640 1 -- 172.21.15.32:0/4240921946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad81527c0 con 0x7f7ad8151a50 2024-04-12T19:51:14.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.495+0000 7f7ad7e00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 0x7f7ad8153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7add400640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ad815d6d0 tx=0x7f7ad000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0016620 con 0x7f7ad8151a50 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad001b070 con 0x7f7ad8151a50 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad8152aa0 con 0x7f7ad8151a50 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0016a20 con 0x7f7ad8151a50 2024-04-12T19:51:14.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.496+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad8152d80 con 0x7f7ad8151a50 2024-04-12T19:51:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.497+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad80755c0 con 0x7f7ad8151a50 2024-04-12T19:51:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.498+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad0016bc0 con 0x7f7ad8151a50 2024-04-12T19:51:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.499+0000 7f7ad4c00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.499+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f7ad008c9d0 con 0x7f7ad8151a50 2024-04-12T19:51:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.499+0000 7f7adca00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.500+0000 7f7adca00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 0x7f7ab806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7acc036000 tx=0x7f7acc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.504+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ad005ad40 con 0x7f7ad8151a50 2024-04-12T19:51:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.709+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ad80b0d30 con 0x7f7ad8151a50 2024-04-12T19:51:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.710+0000 7f7ad4c00640 1 -- 172.21.15.32:0/4240921946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f7ad005a640 con 0x7f7ad8151a50 2024-04-12T19:51:14.711 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:14.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 msgr2=0x7f7ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 0x7f7ab806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7acc036000 tx=0x7f7acc030000 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 msgr2=0x7f7ad8151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ad815d6d0 tx=0x7f7ad000f520 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 shutdown_connections 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7ab806c8f0 0x7f7ab806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad8151a50 0x7f7ad8151ed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad81534e0 0x7f7ad8153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 --2- 172.21.15.32:0/4240921946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ac00039b0 0x7f7ad8141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 >> 172.21.15.32:0/4240921946 conn(0x7f7ad806b930 msgr2=0x7f7ad8156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 shutdown_connections 2024-04-12T19:51:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.713+0000 7f7adde00640 1 -- 172.21.15.32:0/4240921946 wait complete. 2024-04-12T19:51:14.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:14.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.964+0000 7f19e6e00640 1 Processor -- start 2024-04-12T19:51:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.964+0000 7f19e6e00640 1 -- start start 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19e0159390 0x7f19e015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e00b5c90 con 0x7f19e0158fb0 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e00b4670 con 0x7f19e006ffb0 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f19e00b38b0 con 0x7f19e0159390 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19e0159390 0x7f19e015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19e0159390 0x7f19e015fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58564/0 (socket says 172.21.15.32:58564) 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e6400640 1 -- 172.21.15.32:0/1415673539 learned_addr learned my addr 172.21.15.32:0/1415673539 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.966+0000 7f19e5a00640 1 --1- 172.21.15.32:0/1415673539 >> v1:172.21.15.32:6789/0 conn(0x7f19e0158fb0 0x7f19e006f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.967+0000 7f19e6400640 1 -- 172.21.15.32:0/1415673539 >> v1:172.21.15.169:6789/0 conn(0x7f19e006ffb0 legacy=0x7f19e015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.967+0000 7f19e6400640 1 -- 172.21.15.32:0/1415673539 >> v1:172.21.15.32:6789/0 conn(0x7f19e0158fb0 legacy=0x7f19e006f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.967+0000 7f19e6400640 1 -- 172.21.15.32:0/1415673539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e00b3b90 con 0x7f19e0159390 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.967+0000 7f19dfe00640 1 -- 172.21.15.32:0/1415673539 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2578941110 0 0) 0x7f19e00b5c90 con 0x7f19e0158fb0 2024-04-12T19:51:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.969+0000 7f19e6400640 1 --2- 172.21.15.32:0/1415673539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0159390 0x7f19e015fb50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009e50 tx=0x7f19d802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b41adeb7cfd2cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19dfe00640 1 -- 172.21.15.32:0/1415673539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d802cb20 con 0x7f19e0159390 2024-04-12T19:51:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19dfe00640 1 -- 172.21.15.32:0/1415673539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d802ccc0 con 0x7f19e0159390 2024-04-12T19:51:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19dfe00640 1 -- 172.21.15.32:0/1415673539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8034610 con 0x7f19e0159390 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19e6e00640 1 -- 172.21.15.32:0/1415673539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0159390 msgr2=0x7f19e015fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19e6e00640 1 --2- 172.21.15.32:0/1415673539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0159390 0x7f19e015fb50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009e50 tx=0x7f19d802bf80 comp rx=0 tx=0).stop 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19e6e00640 1 -- 172.21.15.32:0/1415673539 shutdown_connections 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19e6e00640 1 --2- 172.21.15.32:0/1415673539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0159390 0x7f19e015fb50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.971+0000 7f19e6e00640 1 -- 172.21.15.32:0/1415673539 >> 172.21.15.32:0/1415673539 conn(0x7f19e00b23a0 msgr2=0x7f19e00b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.972+0000 7f19e6e00640 1 -- 172.21.15.32:0/1415673539 shutdown_connections 2024-04-12T19:51:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.972+0000 7f19e6e00640 1 -- 172.21.15.32:0/1415673539 wait complete. 2024-04-12T19:51:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.973+0000 7f19e6e00640 1 Processor -- start 2024-04-12T19:51:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.974+0000 7f19e6e00640 1 -- start start 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.974+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e006ffb0 0x7f19e015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0159390 0x7f19e0157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006e910 con 0x7f19e0158fb0 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19e006e790 con 0x7f19e0159390 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19e006ea90 con 0x7f19e006ffb0 2024-04-12T19:51:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58570/0 (socket says 172.21.15.32:58570) 2024-04-12T19:51:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 -- 172.21.15.32:0/3977648873 learned_addr learned my addr 172.21.15.32:0/3977648873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e6400640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0159390 0x7f19e0157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5a00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e006ffb0 0x7f19e015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 -- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e006ffb0 msgr2=0x7f19e015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e006ffb0 0x7f19e015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 -- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0159390 msgr2=0x7f19e0157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0159390 0x7f19e0157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.975+0000 7f19e5000640 1 -- 172.21.15.32:0/3977648873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e0157890 con 0x7f19e0158fb0 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19e5000640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c450 tx=0x7f19cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0052d0 con 0x7f19e0158fb0 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc005470 con 0x7f19e0158fb0 2024-04-12T19:51:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e0157b70 con 0x7f19e0158fb0 2024-04-12T19:51:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc012870 con 0x7f19e0158fb0 2024-04-12T19:51:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.976+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e0158130 con 0x7f19e0158fb0 2024-04-12T19:51:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.977+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00755c0 con 0x7f19e0158fb0 2024-04-12T19:51:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.978+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19cc012a10 con 0x7f19e0158fb0 2024-04-12T19:51:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.979+0000 7f19dd600640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.979+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f19cc08b8a0 con 0x7f19e0158fb0 2024-04-12T19:51:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.980+0000 7f19e5a00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.980+0000 7f19e5a00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000f60 tx=0x7f19d4007b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:14.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:14.984+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19cc059b80 con 0x7f19e0158fb0 2024-04-12T19:51:15.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.189+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19e00b2780 con 0x7f19e0158fb0 2024-04-12T19:51:15.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.190+0000 7f19dd600640 1 -- 172.21.15.32:0/3977648873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f19cc059480 con 0x7f19e0158fb0 2024-04-12T19:51:15.190 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.192+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 msgr2=0x7f19c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.192+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000f60 tx=0x7f19d4007b90 comp rx=0 tx=0).stop 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.192+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 msgr2=0x7f19e0156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c450 tx=0x7f19cc00c910 comp rx=0 tx=0).stop 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 shutdown_connections 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e0159390 0x7f19e0157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19e0158fb0 0x7f19e0156b10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 --2- 172.21.15.32:0/3977648873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19e006ffb0 0x7f19e015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 >> 172.21.15.32:0/3977648873 conn(0x7f19e00b23a0 msgr2=0x7f19e00b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 shutdown_connections 2024-04-12T19:51:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.193+0000 7f19e6e00640 1 -- 172.21.15.32:0/3977648873 wait complete. 2024-04-12T19:51:15.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:15.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.437+0000 7fb86b200640 1 Processor -- start 2024-04-12T19:51:15.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.437+0000 7fb86b200640 1 -- start start 2024-04-12T19:51:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.438+0000 7fb86b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb86c162b80 0x7fb86c0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.438+0000 7fb86b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb86c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.438+0000 7fb86b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c072810 con 0x7fb86c162f60 2024-04-12T19:51:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.438+0000 7fb86b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb86c0b49c0 con 0x7fb86c162b80 2024-04-12T19:51:15.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.438+0000 7fb86b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c0b4ca0 con 0x7fb86c0b4390 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb86a800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb86c162f60 0x7fb86c06f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54724/0 (socket says 172.21.15.32:54724) 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb86a800640 1 -- 172.21.15.32:0/485421715 learned_addr learned my addr 172.21.15.32:0/485421715 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb869e00640 1 --2- 172.21.15.32:0/485421715 >> v2:172.21.15.169:3300/0 conn(0x7fb86c162b80 0x7fb86c0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb869400640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb86c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227295729 0 0) 0x7fb86c072810 con 0x7fb86c162f60 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.439+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb850003520 con 0x7fb86c162f60 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 144561568 0 0) 0x7fb850003520 con 0x7fb86c162f60 2024-04-12T19:51:15.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 msgr2=0x7fb86c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb86c071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 >> v2:172.21.15.169:3300/0 conn(0x7fb86c162b80 msgr2=0x7fb86c0b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 --2- 172.21.15.32:0/485421715 >> v2:172.21.15.169:3300/0 conn(0x7fb86c162b80 0x7fb86c0b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c0b4f80 con 0x7fb86c162f60 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fb8640019f0 con 0x7fb86c162f60 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 >> v1:172.21.15.32:6789/0 conn(0x7fb86c162f60 legacy=0x7fb86c06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb850004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb850004d40 0x7fb8500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0b49c0 con 0x7fb8500039b0 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0b4ca0 con 0x7fb850004d40 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb85000b9c0 con 0x7fb86c0b4390 2024-04-12T19:51:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.440+0000 7fb869e00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb86a800640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb850004d40 0x7fb8500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869400640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb850004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869e00640 1 -- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 msgr2=0x7fb850004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869e00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c0b4390 0x7fb850004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869e00640 1 -- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb850004d40 msgr2=0x7fb8500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869e00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb850004d40 0x7fb8500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.441+0000 7fb869e00640 1 -- 172.21.15.32:0/485421715 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb850003520 con 0x7fb8500039b0 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.442+0000 7fb869e00640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009ac0 tx=0x7fb850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ea210ab2974621 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.442+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c039070 con 0x7fb8500039b0 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.442+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c02c710 con 0x7fb8500039b0 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.442+0000 7fb868a00640 1 -- 172.21.15.32:0/485421715 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c009690 con 0x7fb8500039b0 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 -- 172.21.15.32:0/485421715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 msgr2=0x7fb850003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 --2- 172.21.15.32:0/485421715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009ac0 tx=0x7fb850005830 comp rx=0 tx=0).stop 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86a800640 1 -- 172.21.15.32:0/485421715 reap_dead start 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 -- 172.21.15.32:0/485421715 shutdown_connections 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 -- 172.21.15.32:0/485421715 >> 172.21.15.32:0/485421715 conn(0x7fb86c06b930 msgr2=0x7fb86c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 -- 172.21.15.32:0/485421715 shutdown_connections 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.443+0000 7fb86b200640 1 -- 172.21.15.32:0/485421715 wait complete. 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.445+0000 7fb86b200640 1 Processor -- start 2024-04-12T19:51:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.445+0000 7fb86b200640 1 -- start start 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 0x7fb86c14c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 0x7fb86c165d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c06faa0 con 0x7fb8500039b0 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c06f920 con 0x7fb86c155590 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb86b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c071ee0 con 0x7fb86c1658e0 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb869e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb869e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58602/0 (socket says 172.21.15.32:58602) 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.446+0000 7fb869e00640 1 -- 172.21.15.32:0/62192765 learned_addr learned my addr 172.21.15.32:0/62192765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869400640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 0x7fb86c14c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb86a800640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 0x7fb86c165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 -- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 msgr2=0x7fb86c165d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 0x7fb86c165d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 -- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 msgr2=0x7fb86c14c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 0x7fb86c14c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 -- 172.21.15.32:0/62192765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c14cb30 con 0x7fb8500039b0 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869400640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 0x7fb86c14c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb86a800640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 0x7fb86c165d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb869e00640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb85c037040 tx=0x7fb85c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c042020 con 0x7fb8500039b0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.447+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c03a040 con 0x7fb8500039b0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.448+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c039070 con 0x7fb8500039b0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.448+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c163bb0 con 0x7fb8500039b0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.448+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c164170 con 0x7fb8500039b0 2024-04-12T19:51:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.449+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c06c8c0 con 0x7fb8500039b0 2024-04-12T19:51:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.450+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb85c045020 con 0x7fb8500039b0 2024-04-12T19:51:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.451+0000 7fb862000640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.451+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fb85c0ad3d0 con 0x7fb8500039b0 2024-04-12T19:51:15.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.451+0000 7fb869400640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.452+0000 7fb869400640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb86c165670 tx=0x7fb858008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.455+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb85c04c030 con 0x7fb8500039b0 2024-04-12T19:51:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.652+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb86c0755c0 con 0x7fb8500039b0 2024-04-12T19:51:15.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.653+0000 7fb862000640 1 -- 172.21.15.32:0/62192765 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fb85c07b740 con 0x7fb8500039b0 2024-04-12T19:51:15.654 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 msgr2=0x7fb84c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb86c165670 tx=0x7fb858008040 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 msgr2=0x7fb86c163440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb85c037040 tx=0x7fb85c004730 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 shutdown_connections 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb86c1658e0 0x7fb86c165d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb86c155590 0x7fb86c14c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 --2- 172.21.15.32:0/62192765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8500039b0 0x7fb86c163440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 >> 172.21.15.32:0/62192765 conn(0x7fb86c06b930 msgr2=0x7fb86c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:15.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 shutdown_connections 2024-04-12T19:51:15.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.656+0000 7fb86b200640 1 -- 172.21.15.32:0/62192765 wait complete. 2024-04-12T19:51:15.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.901+0000 7fb691e00640 1 Processor -- start 2024-04-12T19:51:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.901+0000 7fb691e00640 1 -- start start 2024-04-12T19:51:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb691e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb68c145170 0x7fb68c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb691e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb68c0b4410 con 0x7fb68c06f260 2024-04-12T19:51:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb691e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb68c0b46f0 con 0x7fb68c0ce070 2024-04-12T19:51:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb691e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb68c145550 con 0x7fb68c145170 2024-04-12T19:51:15.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb68aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb68c145170 0x7fb68c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb68aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb68c145170 0x7fb68c0cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58610/0 (socket says 172.21.15.32:58610) 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.903+0000 7fb68aa00640 1 -- 172.21.15.32:0/1566759603 learned_addr learned my addr 172.21.15.32:0/1566759603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.904+0000 7fb68aa00640 1 -- 172.21.15.32:0/1566759603 >> v1:172.21.15.169:6789/0 conn(0x7fb68c0ce070 legacy=0x7fb68c0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.904+0000 7fb68aa00640 1 -- 172.21.15.32:0/1566759603 >> v1:172.21.15.32:6789/0 conn(0x7fb68c06f260 legacy=0x7fb68c06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.904+0000 7fb68aa00640 1 -- 172.21.15.32:0/1566759603 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c0b49d0 con 0x7fb68c145170 2024-04-12T19:51:15.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.904+0000 7fb689600640 1 -- 172.21.15.32:0/1566759603 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160001837 0 0) 0x7fb68c0b4410 con 0x7fb68c06f260 2024-04-12T19:51:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.905+0000 7fb68aa00640 1 --2- 172.21.15.32:0/1566759603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c145170 0x7fb68c0cdb30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb68c1456d0 tx=0x7fb67c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de1b7152426b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.905+0000 7fb689600640 1 -- 172.21.15.32:0/1566759603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c02f840 con 0x7fb68c145170 2024-04-12T19:51:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb689600640 1 -- 172.21.15.32:0/1566759603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c02f9e0 con 0x7fb68c145170 2024-04-12T19:51:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb689600640 1 -- 172.21.15.32:0/1566759603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c02fdb0 con 0x7fb68c145170 2024-04-12T19:51:15.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 -- 172.21.15.32:0/1566759603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c145170 msgr2=0x7fb68c0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 --2- 172.21.15.32:0/1566759603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c145170 0x7fb68c0cdb30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb68c1456d0 tx=0x7fb67c02ba20 comp rx=0 tx=0).stop 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 -- 172.21.15.32:0/1566759603 shutdown_connections 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 --2- 172.21.15.32:0/1566759603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c145170 0x7fb68c0cdb30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 -- 172.21.15.32:0/1566759603 >> 172.21.15.32:0/1566759603 conn(0x7fb68c0b2460 msgr2=0x7fb68c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 -- 172.21.15.32:0/1566759603 shutdown_connections 2024-04-12T19:51:15.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.906+0000 7fb691e00640 1 -- 172.21.15.32:0/1566759603 wait complete. 2024-04-12T19:51:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.908+0000 7fb691e00640 1 Processor -- start 2024-04-12T19:51:15.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.908+0000 7fb691e00640 1 -- start start 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 0x7fb68c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 0x7fb68c0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0b5bc0 con 0x7fb68c06f260 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0b4f90 con 0x7fb68c0ce070 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.909+0000 7fb691e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c0b4030 con 0x7fb68c145170 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58618/0 (socket says 172.21.15.32:58618) 2024-04-12T19:51:15.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 -- 172.21.15.32:0/104159055 learned_addr learned my addr 172.21.15.32:0/104159055 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68a000640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 0x7fb68c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68b400640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 0x7fb68c0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 -- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 msgr2=0x7fb68c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 0x7fb68c0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 -- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 msgr2=0x7fb68c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 0x7fb68c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68aa00640 1 -- 172.21.15.32:0/104159055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c13ce20 con 0x7fb68c06f260 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68a000640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 0x7fb68c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:15.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.910+0000 7fb68b400640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 0x7fb68c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb68aa00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb67c02ba60 tx=0x7fb67c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c041400 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c046070 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0417e0 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c1e0c30 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.911+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c1e11f0 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.912+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68c068c80 con 0x7fb68c06f260 2024-04-12T19:51:15.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.913+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb67c040070 con 0x7fb68c06f260 2024-04-12T19:51:15.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.914+0000 7fb683400640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 0x7fb66c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:15.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.914+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fb67c0ae470 con 0x7fb68c06f260 2024-04-12T19:51:15.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.914+0000 7fb68a000640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 0x7fb66c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:15.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.915+0000 7fb68a000640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 0x7fb66c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0b23f0 tx=0x7fb678009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:15.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:15.919+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb67c033030 con 0x7fb68c06f260 2024-04-12T19:51:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.122+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb68c0b0550 con 0x7fb68c06f260 2024-04-12T19:51:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.123+0000 7fb683400640 1 -- 172.21.15.32:0/104159055 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb67c07c750 con 0x7fb68c06f260 2024-04-12T19:51:16.124 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 msgr2=0x7fb66c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 0x7fb66c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0b23f0 tx=0x7fb678009040 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 msgr2=0x7fb68c13c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb67c02ba60 tx=0x7fb67c004730 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 shutdown_connections 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb66c06c920 0x7fb66c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb68c145170 0x7fb68c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb68c0ce070 0x7fb68c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 --2- 172.21.15.32:0/104159055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb68c06f260 0x7fb68c13c8e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.126+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 >> 172.21.15.32:0/104159055 conn(0x7fb68c0b2460 msgr2=0x7fb68c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.127+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 shutdown_connections 2024-04-12T19:51:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.127+0000 7fb691e00640 1 -- 172.21.15.32:0/104159055 wait complete. 2024-04-12T19:51:16.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T19:51:16.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.363+0000 7f10a1e00640 1 Processor -- start 2024-04-12T19:51:16.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.364+0000 7f10a1e00640 1 -- start start 2024-04-12T19:51:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f109c0d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f10a1e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109c0d32c0 con 0x7f109c0d1300 2024-04-12T19:51:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f10a1e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109c0d35a0 con 0x7f109c0d2c30 2024-04-12T19:51:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f10a1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f109c0d3090 con 0x7f109c15ec40 2024-04-12T19:51:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f109b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f109c0d1300 0x7f109c0b78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54742/0 (socket says 172.21.15.32:54742) 2024-04-12T19:51:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.365+0000 7f109b400640 1 -- 172.21.15.32:0/3893413930 learned_addr learned my addr 172.21.15.32:0/3893413930 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f109aa00640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f109c0d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788592304 0 0) 0x7f109c0d32c0 con 0x7f109c0d1300 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1080003520 con 0x7f109c0d1300 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2340802305 0 0) 0x7f1080003520 con 0x7f109c0d1300 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 msgr2=0x7f109c0d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f109c0d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 >> v1:172.21.15.169:6789/0 conn(0x7f109c0d2c30 legacy=0x7f109c0ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c0d3880 con 0x7f109c0d1300 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.366+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f1094001670 con 0x7f109c0d1300 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 >> v1:172.21.15.32:6789/0 conn(0x7f109c0d1300 legacy=0x7f109c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f10800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080003f60 0x7f1080004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1080004d40 0x7f10800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f108000ba70 con 0x7f1080004d40 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f108000bc20 con 0x7f1080003f60 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f108000be20 con 0x7f109c15ec40 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1080004d40 0x7f10800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109a000640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080003f60 0x7f1080004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109aa00640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f10800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 -- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 msgr2=0x7f10800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f10800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 -- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080003f60 msgr2=0x7f1080004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080003f60 0x7f1080004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109b400640 1 -- 172.21.15.32:0/3893413930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1080003520 con 0x7f1080004d40 2024-04-12T19:51:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.367+0000 7f109aa00640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c15ec40 0x7f10800039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:16.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f109b400640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1080004d40 0x7f10800051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f109400c240 tx=0x7f1080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cf095bb3a3d734 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094007d10 con 0x7f1080004d40 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1094030070 con 0x7f1080004d40 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f1099600640 1 -- 172.21.15.32:0/3893413930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109402cdc0 con 0x7f1080004d40 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f10a1e00640 1 -- 172.21.15.32:0/3893413930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1080004d40 msgr2=0x7f10800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f10a1e00640 1 --2- 172.21.15.32:0/3893413930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1080004d40 0x7f10800051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f109400c240 tx=0x7f1080005830 comp rx=0 tx=0).stop 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f109b400640 1 -- 172.21.15.32:0/3893413930 reap_dead start 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f10a1e00640 1 -- 172.21.15.32:0/3893413930 shutdown_connections 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.369+0000 7f10a1e00640 1 -- 172.21.15.32:0/3893413930 >> 172.21.15.32:0/3893413930 conn(0x7f109c073690 msgr2=0x7f109c06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.370+0000 7f10a1e00640 1 -- 172.21.15.32:0/3893413930 shutdown_connections 2024-04-12T19:51:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.370+0000 7f10a1e00640 1 -- 172.21.15.32:0/3893413930 wait complete. 2024-04-12T19:51:16.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.371+0000 7f10a1e00640 1 Processor -- start 2024-04-12T19:51:16.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 -- start start 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 0x7f109c0b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0cdbd0 0x7f109c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f109c0d4360 con 0x7f109c0b9d10 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f109c0d3e40 con 0x7f1080004d40 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.372+0000 7f10a1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f109c0d44e0 con 0x7f109c0cdbd0 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58650/0 (socket says 172.21.15.32:58650) 2024-04-12T19:51:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 -- 172.21.15.32:0/2125697606 learned_addr learned my addr 172.21.15.32:0/2125697606 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109a000640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0cdbd0 0x7f109c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109aa00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 0x7f109c0b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 -- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0cdbd0 msgr2=0x7f109c0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0cdbd0 0x7f109c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 -- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 msgr2=0x7f109c0b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 0x7f109c0b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109b400640 1 -- 172.21.15.32:0/2125697606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c1f4540 con 0x7f109c0b9d10 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.373+0000 7f109aa00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 0x7f109c0b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f109b400640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f109400c360 tx=0x7f109c0d3490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094007d10 con 0x7f109c0b9d10 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10940049f0 con 0x7f109c0b9d10 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094004dc0 con 0x7f109c0b9d10 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109c1f4820 con 0x7f109c0b9d10 2024-04-12T19:51:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.374+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109c1f4de0 con 0x7f109c0b9d10 2024-04-12T19:51:16.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.375+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109c068c80 con 0x7f109c0b9d10 2024-04-12T19:51:16.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.376+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1094007070 con 0x7f109c0b9d10 2024-04-12T19:51:16.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.377+0000 7f1093400640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 0x7f107806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:16.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.377+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f10940ac420 con 0x7f109c0b9d10 2024-04-12T19:51:16.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.377+0000 7f109aa00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 0x7f107806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:16.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.378+0000 7f109aa00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 0x7f107806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f108c007520 tx=0x7f108c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:16.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.382+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f109407a830 con 0x7f109c0b9d10 2024-04-12T19:51:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.598+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f109c0d18e0 con 0x7f109c0b9d10 2024-04-12T19:51:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.599+0000 7f1093400640 1 -- 172.21.15.32:0/2125697606 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f109407a130 con 0x7f109c0b9d10 2024-04-12T19:51:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 msgr2=0x7f107806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 0x7f107806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f108c007520 tx=0x7f108c00d040 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 msgr2=0x7f109c0ba190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f109400c360 tx=0x7f109c0d3490 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 shutdown_connections 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f107806c920 0x7f107806ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f109c0b9d10 0x7f109c0ba190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0cdbd0 0x7f109c0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 --2- 172.21.15.32:0/2125697606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1080004d40 0x7f109c0b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 >> 172.21.15.32:0/2125697606 conn(0x7f109c073690 msgr2=0x7f109c15cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 shutdown_connections 2024-04-12T19:51:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:16.602+0000 7f10a1e00640 1 -- 172.21.15.32:0/2125697606 wait complete. 2024-04-12T19:51:16.621 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-12T19:51:16.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:16.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-12T19:51:16.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:51:16.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:16.722 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-12T19:51:16.771 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:16.771 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:16.829 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:16.829 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:16.883 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:16.883 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:16.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.166+0000 7f7f2c800640 1 Processor -- start 2024-04-12T19:51:17.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.167+0000 7f7f2c800640 1 -- start start 2024-04-12T19:51:17.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.168+0000 7f7f2c800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 0x7f7f240d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.168+0000 7f7f2c800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240b3470 0x7f7f240b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.168+0000 7f7f2c800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f240d2d90 con 0x7f7f2415eac0 2024-04-12T19:51:17.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.168+0000 7f7f2c800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240d1cd0 con 0x7f7f240b3470 2024-04-12T19:51:17.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.168+0000 7f7f2c800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7f240d3070 con 0x7f7f240d1300 2024-04-12T19:51:17.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f29e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f2415eac0 0x7f7f240d3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54746/0 (socket says 172.21.15.32:54746) 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f2a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f240b3470 0x7f7f240b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f29e00640 1 -- 172.21.15.32:0/2488853982 learned_addr learned my addr 172.21.15.32:0/2488853982 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f29400640 1 --2- 172.21.15.32:0/2488853982 >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 0x7f7f240d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f2a800640 1 -- 172.21.15.32:0/2488853982 >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 msgr2=0x7f7f240d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f2a800640 1 --2- 172.21.15.32:0/2488853982 >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 0x7f7f240d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f2a800640 1 -- 172.21.15.32:0/2488853982 >> v1:172.21.15.32:6789/0 conn(0x7f7f2415eac0 legacy=0x7f7f240d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f28a00640 1 -- 172.21.15.32:0/2488853982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569048509 0 0) 0x7f7f240d2d90 con 0x7f7f2415eac0 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f2a800640 1 -- 172.21.15.32:0/2488853982 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240d3350 con 0x7f7f240b3470 2024-04-12T19:51:17.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.169+0000 7f7f29400640 1 --2- 172.21.15.32:0/2488853982 >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 0x7f7f240d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:17.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.171+0000 7f7f2a800640 1 --2- 172.21.15.32:0/2488853982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240b3470 0x7f7f240b3880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f240d20b0 tx=0x7f7f2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc6b84fb7c4a275 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.171+0000 7f7f28a00640 1 -- 172.21.15.32:0/2488853982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20039070 con 0x7f7f240b3470 2024-04-12T19:51:17.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.171+0000 7f7f28a00640 1 -- 172.21.15.32:0/2488853982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2002cc80 con 0x7f7f240b3470 2024-04-12T19:51:17.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.171+0000 7f7f28a00640 1 -- 172.21.15.32:0/2488853982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20034660 con 0x7f7f240b3470 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 -- 172.21.15.32:0/2488853982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240b3470 msgr2=0x7f7f240b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 --2- 172.21.15.32:0/2488853982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240b3470 0x7f7f240b3880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f240d20b0 tx=0x7f7f2002bed0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 -- 172.21.15.32:0/2488853982 shutdown_connections 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 --2- 172.21.15.32:0/2488853982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f240b3470 0x7f7f240b3880 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 --2- 172.21.15.32:0/2488853982 >> v2:172.21.15.169:3300/0 conn(0x7f7f240d1300 0x7f7f240d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 -- 172.21.15.32:0/2488853982 >> 172.21.15.32:0/2488853982 conn(0x7f7f24073690 msgr2=0x7f7f2406cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 -- 172.21.15.32:0/2488853982 shutdown_connections 2024-04-12T19:51:17.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.172+0000 7f7f2c800640 1 -- 172.21.15.32:0/2488853982 wait complete. 2024-04-12T19:51:17.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.173+0000 7f7f2c800640 1 Processor -- start 2024-04-12T19:51:17.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.174+0000 7f7f2c800640 1 -- start start 2024-04-12T19:51:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.174+0000 7f7f2c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 0x7f7f240ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 0x7f7f241618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2c800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2c800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2415eea0 con 0x7f7f2415eac0 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f240d3910 con 0x7f7f240d1300 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7f240d4590 con 0x7f7f240b3470 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58674/0 (socket says 172.21.15.32:58674) 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 -- 172.21.15.32:0/1355967551 learned_addr learned my addr 172.21.15.32:0/1355967551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f29400640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 0x7f7f241618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f29e00640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 0x7f7f240ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 -- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 msgr2=0x7f7f240ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 0x7f7f240ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 -- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 msgr2=0x7f7f241618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 0x7f7f241618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f2a800640 1 -- 172.21.15.32:0/1355967551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f240ab590 con 0x7f7f2415eac0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f29400640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 0x7f7f241618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.175+0000 7f7f29e00640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 0x7f7f240ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f2a800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f20037040 tx=0x7f7f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20039050 con 0x7f7f2415eac0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2003a040 con 0x7f7f2415eac0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2003e610 con 0x7f7f2415eac0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f240ab870 con 0x7f7f2415eac0 2024-04-12T19:51:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.176+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2414ba00 con 0x7f7f2415eac0 2024-04-12T19:51:17.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.178+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24068c80 con 0x7f7f2415eac0 2024-04-12T19:51:17.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.179+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f20047020 con 0x7f7f2415eac0 2024-04-12T19:51:17.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.180+0000 7f7f1e000640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 0x7f7f0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.180+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f7f200adf30 con 0x7f7f2415eac0 2024-04-12T19:51:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.180+0000 7f7f29e00640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 0x7f7f0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.181+0000 7f7f29e00640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 0x7f7f0806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f60 tx=0x7f7f18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.184+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f2004e020 con 0x7f7f2415eac0 2024-04-12T19:51:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.390+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f240cf2d0 con 0x7f7f2415eac0 2024-04-12T19:51:17.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.393+0000 7f7f1e000640 1 -- 172.21.15.32:0/1355967551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f7f2007c3d0 con 0x7f7f2415eac0 2024-04-12T19:51:17.394 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:17.394 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 msgr2=0x7f7f0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 0x7f7f0806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f60 tx=0x7f7f18009040 comp rx=0 tx=0).stop 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 msgr2=0x7f7f24160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f20037040 tx=0x7f7f20004a40 comp rx=0 tx=0).stop 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 shutdown_connections 2024-04-12T19:51:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7f0806c8f0 0x7f7f0806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2415eac0 0x7f7f24160ad0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f240d1300 0x7f7f241618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 --2- 172.21.15.32:0/1355967551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f240b3470 0x7f7f240ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 >> 172.21.15.32:0/1355967551 conn(0x7f7f24073690 msgr2=0x7f7f240cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 shutdown_connections 2024-04-12T19:51:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.396+0000 7f7f2c800640 1 -- 172.21.15.32:0/1355967551 wait complete. 2024-04-12T19:51:17.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:17.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:17.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.644+0000 7f56c6600640 1 Processor -- start 2024-04-12T19:51:17.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.644+0000 7f56c6600640 1 -- start start 2024-04-12T19:51:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00d26b0 0x7f56c00b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f56c00b7e80 0x7f56c00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c6600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c00cfef0 con 0x7f56c0161f80 2024-04-12T19:51:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f56c00d2bb0 con 0x7f56c00b7e80 2024-04-12T19:51:17.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56c00d2e90 con 0x7f56c00d26b0 2024-04-12T19:51:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c5c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f56c00b7e80 0x7f56c00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.646+0000 7f56c5c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f56c00b7e80 0x7f56c00d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:48016/0 (socket says 172.21.15.32:48016) 2024-04-12T19:51:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56c5c00640 1 -- 172.21.15.32:0/2889456965 learned_addr learned my addr 172.21.15.32:0/2889456965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56bfe00640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00d26b0 0x7f56c00b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56c5c00640 1 -- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00d26b0 msgr2=0x7f56c00b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:17.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56c5c00640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00d26b0 0x7f56c00b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56c5c00640 1 -- 172.21.15.32:0/2889456965 >> v1:172.21.15.169:6789/0 conn(0x7f56c0161f80 legacy=0x7f56c00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.647+0000 7f56c5c00640 1 -- 172.21.15.32:0/2889456965 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c00d3170 con 0x7f56c00b7e80 2024-04-12T19:51:17.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.649+0000 7f56c5c00640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c00b7e80 0x7f56c00d1580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009e50 tx=0x7f56b802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7ddc1da4066025 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.649+0000 7f56bf400640 1 -- 172.21.15.32:0/2889456965 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b802cb00 con 0x7f56c00b7e80 2024-04-12T19:51:17.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.649+0000 7f56bf400640 1 -- 172.21.15.32:0/2889456965 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b802cca0 con 0x7f56c00b7e80 2024-04-12T19:51:17.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.649+0000 7f56bf400640 1 -- 172.21.15.32:0/2889456965 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8034610 con 0x7f56c00b7e80 2024-04-12T19:51:17.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 -- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c00b7e80 msgr2=0x7f56c00d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c00b7e80 0x7f56c00d1580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009e50 tx=0x7f56b802bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:17.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 -- 172.21.15.32:0/2889456965 shutdown_connections 2024-04-12T19:51:17.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c00b7e80 0x7f56c00d1580 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 --2- 172.21.15.32:0/2889456965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00d26b0 0x7f56c00b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 -- 172.21.15.32:0/2889456965 >> 172.21.15.32:0/2889456965 conn(0x7f56c006ce40 msgr2=0x7f56c006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:17.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 -- 172.21.15.32:0/2889456965 shutdown_connections 2024-04-12T19:51:17.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.650+0000 7f56c6600640 1 -- 172.21.15.32:0/2889456965 wait complete. 2024-04-12T19:51:17.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.652+0000 7f56c6600640 1 Processor -- start 2024-04-12T19:51:17.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.652+0000 7f56c6600640 1 -- start start 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00b7e80 0x7f56c01f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 0x7f56c01f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56c00d1ac0 con 0x7f56c00d26b0 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0162360 con 0x7f56c0161f80 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56c6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56c00d1c40 con 0x7f56c00b7e80 2024-04-12T19:51:17.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.653+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58686/0 (socket says 172.21.15.32:58686) 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 -- 172.21.15.32:0/2718927377 learned_addr learned my addr 172.21.15.32:0/2718927377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56c5200640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00b7e80 0x7f56c01f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 -- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00b7e80 msgr2=0x7f56c01f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00b7e80 0x7f56c01f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56c5c00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 0x7f56c01f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 -- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 msgr2=0x7f56c01f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 0x7f56c01f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 -- 172.21.15.32:0/2718927377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c01f5500 con 0x7f56c00d26b0 2024-04-12T19:51:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56c5c00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 0x7f56c01f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.654+0000 7f56bfe00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00e9e0 tx=0x7f56ac00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.655+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac017070 con 0x7f56c00d26b0 2024-04-12T19:51:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.655+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac00c9d0 con 0x7f56c00d26b0 2024-04-12T19:51:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.655+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00cdc0 con 0x7f56c00d26b0 2024-04-12T19:51:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.655+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c01f5b00 con 0x7f56c00d26b0 2024-04-12T19:51:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.655+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01f6340 con 0x7f56c00d26b0 2024-04-12T19:51:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.656+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c00755c0 con 0x7f56c00d26b0 2024-04-12T19:51:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.657+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56ac01a070 con 0x7f56c00d26b0 2024-04-12T19:51:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.658+0000 7f56bcc00640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 0x7f56a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.658+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f56ac08bcf0 con 0x7f56c00d26b0 2024-04-12T19:51:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.658+0000 7f56c5200640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 0x7f56a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.659+0000 7f56c5200640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 0x7f56a006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007fd0 tx=0x7f56b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.662+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56ac090070 con 0x7f56c00d26b0 2024-04-12T19:51:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.864+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56c00d2490 con 0x7f56c00d26b0 2024-04-12T19:51:17.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.865+0000 7f56bcc00640 1 -- 172.21.15.32:0/2718927377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f56c01f5b00 con 0x7f56c00d26b0 2024-04-12T19:51:17.865 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 msgr2=0x7f56a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 0x7f56a006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007fd0 tx=0x7f56b0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 msgr2=0x7f56c01f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:17.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00e9e0 tx=0x7f56ac00eea0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 shutdown_connections 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f56a006c8f0 0x7f56a006edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c0161f80 0x7f56c01f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c00d26b0 0x7f56c01f4fc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 --2- 172.21.15.32:0/2718927377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c00b7e80 0x7f56c01f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.868+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 >> 172.21.15.32:0/2718927377 conn(0x7f56c006ce40 msgr2=0x7f56c00cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.869+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 shutdown_connections 2024-04-12T19:51:17.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:17.869+0000 7f56c6600640 1 -- 172.21.15.32:0/2718927377 wait complete. 2024-04-12T19:51:17.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.124+0000 7fcdf2600640 1 Processor -- start 2024-04-12T19:51:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.125+0000 7fcdf2600640 1 -- start start 2024-04-12T19:51:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.126+0000 7fcdf2600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdec15adc0 0x7fcdec0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.126+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.126+0000 7fcdf2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdec15f740 con 0x7fcdec0cf190 2024-04-12T19:51:18.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.126+0000 7fcdf2600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcdec0cd080 con 0x7fcdec15adc0 2024-04-12T19:51:18.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.126+0000 7fcdf2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec06df20 con 0x7fcdec15f2b0 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdec15adc0 0x7fcdec0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdec15adc0 0x7fcdec0b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58702/0 (socket says 172.21.15.32:58702) 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 -- 172.21.15.32:0/2552287032 learned_addr learned my addr 172.21.15.32:0/2552287032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdebe00640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 -- 172.21.15.32:0/2552287032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 msgr2=0x7fcdec0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 -- 172.21.15.32:0/2552287032 >> v1:172.21.15.169:6789/0 conn(0x7fcdec0cf190 legacy=0x7fcdec072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.127+0000 7fcdf1200640 1 -- 172.21.15.32:0/2552287032 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec0cd2d0 con 0x7fcdec15adc0 2024-04-12T19:51:18.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.129+0000 7fcdf1200640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec0b55f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4910dbf3bf6e806e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.129+0000 7fcdeb400640 1 -- 172.21.15.32:0/2552287032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02ca70 con 0x7fcdec15adc0 2024-04-12T19:51:18.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.129+0000 7fcdeb400640 1 -- 172.21.15.32:0/2552287032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc02cc10 con 0x7fcdec15adc0 2024-04-12T19:51:18.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.129+0000 7fcdeb400640 1 -- 172.21.15.32:0/2552287032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0343f0 con 0x7fcdec15adc0 2024-04-12T19:51:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 -- 172.21.15.32:0/2552287032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 msgr2=0x7fcdec0b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec0b55f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02bed0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 -- 172.21.15.32:0/2552287032 shutdown_connections 2024-04-12T19:51:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 --2- 172.21.15.32:0/2552287032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec0b55f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 -- 172.21.15.32:0/2552287032 >> 172.21.15.32:0/2552287032 conn(0x7fcdec06b930 msgr2=0x7fcdec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 -- 172.21.15.32:0/2552287032 shutdown_connections 2024-04-12T19:51:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.130+0000 7fcdf2600640 1 -- 172.21.15.32:0/2552287032 wait complete. 2024-04-12T19:51:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.132+0000 7fcdf2600640 1 Processor -- start 2024-04-12T19:51:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.133+0000 7fcdf2600640 1 -- start start 2024-04-12T19:51:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 0x7fcdec15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec072e80 con 0x7fcdec15adc0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec06e170 con 0x7fcdec0cf190 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdf2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec073000 con 0x7fcdec15f2b0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58712/0 (socket says 172.21.15.32:58712) 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.134+0000 7fcdebe00640 1 -- 172.21.15.32:0/1893541806 learned_addr learned my addr 172.21.15.32:0/1893541806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdf1200640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 0x7fcdec15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdf1c00640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 -- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 msgr2=0x7fcdec144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 -- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 msgr2=0x7fcdec15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 0x7fcdec15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 -- 172.21.15.32:0/1893541806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec145010 con 0x7fcdec15adc0 2024-04-12T19:51:18.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdf1200640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 0x7fcdec15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcdebe00640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800d5f0 tx=0x7fcdd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80125f0 con 0x7fcdec15adc0 2024-04-12T19:51:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.135+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8019070 con 0x7fcdec15adc0 2024-04-12T19:51:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.136+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80129d0 con 0x7fcdec15adc0 2024-04-12T19:51:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.136+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec14aeb0 con 0x7fcdec15adc0 2024-04-12T19:51:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.136+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec14b470 con 0x7fcdec15adc0 2024-04-12T19:51:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.137+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec0755c0 con 0x7fcdec15adc0 2024-04-12T19:51:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.138+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdd8012b70 con 0x7fcdec15adc0 2024-04-12T19:51:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.139+0000 7fcde8c00640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 0x7fcdcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.139+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fcdd808c060 con 0x7fcdec15adc0 2024-04-12T19:51:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.139+0000 7fcdf1200640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 0x7fcdcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.140+0000 7fcdf1200640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 0x7fcdcc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcddc035000 tx=0x7fcddc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.145+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdd805a3d0 con 0x7fcdec15adc0 2024-04-12T19:51:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.348+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcdec06f640 con 0x7fcdec15adc0 2024-04-12T19:51:18.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.349+0000 7fcde8c00640 1 -- 172.21.15.32:0/1893541806 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fcdd8059cd0 con 0x7fcdec15adc0 2024-04-12T19:51:18.349 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 msgr2=0x7fcdcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 0x7fcdcc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcddc035000 tx=0x7fcddc030000 comp rx=0 tx=0).stop 2024-04-12T19:51:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 msgr2=0x7fcdec14d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800d5f0 tx=0x7fcdd800dab0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 shutdown_connections 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcdcc06c8f0 0x7fcdcc06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec15f2b0 0x7fcdec144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdec15adc0 0x7fcdec14d5a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 --2- 172.21.15.32:0/1893541806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0cf190 0x7fcdec15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 >> 172.21.15.32:0/1893541806 conn(0x7fcdec06b930 msgr2=0x7fcdec0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 shutdown_connections 2024-04-12T19:51:18.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.352+0000 7fcdf2600640 1 -- 172.21.15.32:0/1893541806 wait complete. 2024-04-12T19:51:18.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.600+0000 7f668f000640 1 Processor -- start 2024-04-12T19:51:18.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.601+0000 7f668f000640 1 -- start start 2024-04-12T19:51:18.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66880ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668f000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66880cfc30 con 0x7f668815ed30 2024-04-12T19:51:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66880d35c0 con 0x7f66880d1570 2024-04-12T19:51:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66880ba670 con 0x7f66880cf650 2024-04-12T19:51:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668d200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f668815ed30 0x7f66880b78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54754/0 (socket says 172.21.15.32:54754) 2024-04-12T19:51:18.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.602+0000 7f668d200640 1 -- 172.21.15.32:0/1265342524 learned_addr learned my addr 172.21.15.32:0/1265342524 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f668e600640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66880ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033915314 0 0) 0x7f66880cfc30 con 0x7f668815ed30 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6670003520 con 0x7f668815ed30 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2511791589 0 0) 0x7f6670003520 con 0x7f668815ed30 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 msgr2=0x7f66880ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66880ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 >> v1:172.21.15.169:6789/0 conn(0x7f66880d1570 legacy=0x7f66880d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.603+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66880d38a0 con 0x7f668815ed30 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6678001670 con 0x7f668815ed30 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 >> v1:172.21.15.32:6789/0 conn(0x7f668815ed30 legacy=0x7f66880b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66700039e0 0x7f6670003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6670004800 0x7f6670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66880ba670 con 0x7f6670004800 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f667000bad0 con 0x7f66700039e0 2024-04-12T19:51:18.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f667000bcd0 con 0x7f66880cf650 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668d200640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6670004800 0x7f6670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668dc00640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66700039e0 0x7f6670003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668e600640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668d200640 1 -- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 msgr2=0x7f66700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668d200640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.604+0000 7f668d200640 1 -- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66700039e0 msgr2=0x7f6670003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.605+0000 7f668d200640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66700039e0 0x7f6670003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.605+0000 7f668d200640 1 -- 172.21.15.32:0/1265342524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6670003520 con 0x7f6670004800 2024-04-12T19:51:18.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.605+0000 7f668e600640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66880cf650 0x7f66700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.606+0000 7f668d200640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6670004800 0x7f6670004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f668815f130 tx=0x7f6670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce1e6ab526935b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.606+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678007c20 con 0x7f6670004800 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.606+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6678030070 con 0x7f6670004800 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.606+0000 7f6687e00640 1 -- 172.21.15.32:0/1265342524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678005d30 con 0x7f6670004800 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 -- 172.21.15.32:0/1265342524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6670004800 msgr2=0x7f6670004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 --2- 172.21.15.32:0/1265342524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6670004800 0x7f6670004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f668815f130 tx=0x7f6670005830 comp rx=0 tx=0).stop 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668e600640 1 -- 172.21.15.32:0/1265342524 reap_dead start 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 -- 172.21.15.32:0/1265342524 shutdown_connections 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 -- 172.21.15.32:0/1265342524 >> 172.21.15.32:0/1265342524 conn(0x7f6688073240 msgr2=0x7f6688073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 -- 172.21.15.32:0/1265342524 shutdown_connections 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.607+0000 7f668f000640 1 -- 172.21.15.32:0/1265342524 wait complete. 2024-04-12T19:51:18.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.609+0000 7f668f000640 1 Processor -- start 2024-04-12T19:51:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.609+0000 7f668f000640 1 -- start start 2024-04-12T19:51:18.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 0x7f66881526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6688161d20 0x7f66880cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66880ba130 con 0x7f66880cddb0 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66880cff10 con 0x7f6688161d20 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66880ba2b0 con 0x7f6670004800 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58726/0 (socket says 172.21.15.32:58726) 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668e600640 1 -- 172.21.15.32:0/2577016308 learned_addr learned my addr 172.21.15.32:0/2577016308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668d200640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6688161d20 0x7f66880cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.610+0000 7f668dc00640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 0x7f66881526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 -- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 msgr2=0x7f66881526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 0x7f66881526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 -- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6688161d20 msgr2=0x7f66880cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6688161d20 0x7f66880cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 -- 172.21.15.32:0/2577016308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66880b5c40 con 0x7f66880cddb0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668dc00640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 0x7f66881526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668e600640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66881625c0 tx=0x7f668000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001c070 con 0x7f66880cddb0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6680017400 con 0x7f66880cddb0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800177d0 con 0x7f66880cddb0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.611+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66880b5f20 con 0x7f66880cddb0 2024-04-12T19:51:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.612+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66880b64e0 con 0x7f66880cddb0 2024-04-12T19:51:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.613+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66880755c0 con 0x7f66880cddb0 2024-04-12T19:51:18.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.614+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f668001f070 con 0x7f66880cddb0 2024-04-12T19:51:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.615+0000 7f6685600640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.615+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f668008bdf0 con 0x7f66880cddb0 2024-04-12T19:51:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.615+0000 7f668dc00640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.616+0000 7f668dc00640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 0x7f666c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f667c001630 tx=0x7f667c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.620+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6680012070 con 0x7f66880cddb0 2024-04-12T19:51:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.825+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66880b7440 con 0x7f66880cddb0 2024-04-12T19:51:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.826+0000 7f6685600640 1 -- 172.21.15.32:0/2577016308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f668005a160 con 0x7f66880cddb0 2024-04-12T19:51:18.827 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 msgr2=0x7f666c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 0x7f666c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f667c001630 tx=0x7f667c009040 comp rx=0 tx=0).stop 2024-04-12T19:51:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 msgr2=0x7f66880ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66881625c0 tx=0x7f668000f4a0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 shutdown_connections 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66880cddb0 0x7f66880ce240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6688161d20 0x7f66880cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 --2- 172.21.15.32:0/2577016308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6670004800 0x7f66881526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 >> 172.21.15.32:0/2577016308 conn(0x7f6688073240 msgr2=0x7f668815f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 shutdown_connections 2024-04-12T19:51:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:18.829+0000 7f668f000640 1 -- 172.21.15.32:0/2577016308 wait complete. 2024-04-12T19:51:18.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T19:51:19.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.072+0000 7ff847e00640 1 Processor -- start 2024-04-12T19:51:19.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.073+0000 7ff847e00640 1 -- start start 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff847e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff84815eab0 0x7ff848162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff847e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff8480d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff847e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8480b78f0 con 0x7ff848162b70 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff847e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff8480d2e50 con 0x7ff84815eab0 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff847e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff848163010 con 0x7ff8480d1c60 2024-04-12T19:51:19.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff83fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff848162b70 0x7ff8480d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54756/0 (socket says 172.21.15.32:54756) 2024-04-12T19:51:19.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.074+0000 7ff83fe00640 1 -- 172.21.15.32:0/2859289628 learned_addr learned my addr 172.21.15.32:0/2859289628 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846a00640 1 --2- 172.21.15.32:0/2859289628 >> v2:172.21.15.169:3300/0 conn(0x7ff84815eab0 0x7ff848162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff847400640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff8480d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192554837 0 0) 0x7ff830001110 con 0x7ff848162b70 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff82c003520 con 0x7ff848162b70 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2784783915 0 0) 0x7ff82c003520 con 0x7ff848162b70 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 msgr2=0x7ff8480d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff8480d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 >> v2:172.21.15.169:3300/0 conn(0x7ff84815eab0 msgr2=0x7ff848162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 --2- 172.21.15.32:0/2859289628 >> v2:172.21.15.169:3300/0 conn(0x7ff84815eab0 0x7ff848162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481632f0 con 0x7ff848162b70 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7ff830001840 con 0x7ff848162b70 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 >> v1:172.21.15.32:6789/0 conn(0x7ff848162b70 legacy=0x7ff8480d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c0039b0 0x7ff82c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff82c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.075+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8480d2e50 con 0x7ff82c004d40 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff848163010 con 0x7ff82c0039b0 2024-04-12T19:51:19.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c00b9c0 con 0x7ff8480d1c60 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff83fe00640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff82c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 -- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 msgr2=0x7ff82c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8480d1c60 0x7ff82c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 -- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c0039b0 msgr2=0x7ff82c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c0039b0 0x7ff82c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.076+0000 7ff847400640 1 -- 172.21.15.32:0/2859289628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c003520 con 0x7ff82c004d40 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.077+0000 7ff847400640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff840009f90 tx=0x7ff82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810fdbb06c02e0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840039070 con 0x7ff82c004d40 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84002cb90 con 0x7ff82c004d40 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff846000640 1 -- 172.21.15.32:0/2859289628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840009c10 con 0x7ff82c004d40 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 -- 172.21.15.32:0/2859289628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff82c004d40 msgr2=0x7ff82c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 --2- 172.21.15.32:0/2859289628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff82c004d40 0x7ff82c0051c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff840009f90 tx=0x7ff82c005830 comp rx=0 tx=0).stop 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847400640 1 -- 172.21.15.32:0/2859289628 reap_dead start 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 -- 172.21.15.32:0/2859289628 shutdown_connections 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 -- 172.21.15.32:0/2859289628 >> 172.21.15.32:0/2859289628 conn(0x7ff848073240 msgr2=0x7ff848073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 -- 172.21.15.32:0/2859289628 shutdown_connections 2024-04-12T19:51:19.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.078+0000 7ff847e00640 1 -- 172.21.15.32:0/2859289628 wait complete. 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.080+0000 7ff847e00640 1 Processor -- start 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.080+0000 7ff847e00640 1 -- start start 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c004d40 0x7ff8481f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff84815cf60 0x7ff84815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff8480ba590 con 0x7ff8481f8f80 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff84815ee90 con 0x7ff82c004d40 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.081+0000 7ff847e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8480ba710 con 0x7ff84815cf60 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58736/0 (socket says 172.21.15.32:58736) 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 -- 172.21.15.32:0/3369432180 learned_addr learned my addr 172.21.15.32:0/3369432180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 -- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff84815cf60 msgr2=0x7ff84815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff84815cf60 0x7ff84815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 -- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c004d40 msgr2=0x7ff8481f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c004d40 0x7ff8481f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 -- 172.21.15.32:0/3369432180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84815d920 con 0x7ff8481f8f80 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.082+0000 7ff83fe00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff83000d8b0 tx=0x7ff83000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.083+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830019070 con 0x7ff8481f8f80 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.083+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8300074e0 con 0x7ff8481f8f80 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.083+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300049e0 con 0x7ff8481f8f80 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.083+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8480cd690 con 0x7ff8481f8f80 2024-04-12T19:51:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.083+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84814a960 con 0x7ff8481f8f80 2024-04-12T19:51:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.084+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8480755c0 con 0x7ff8481f8f80 2024-04-12T19:51:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.085+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff83001c070 con 0x7ff8481f8f80 2024-04-12T19:51:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.086+0000 7ff83ea00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 0x7ff82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.086+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7ff83008b720 con 0x7ff8481f8f80 2024-04-12T19:51:19.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.086+0000 7ff846a00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 0x7ff82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.087+0000 7ff846a00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 0x7ff82406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff838006a90 tx=0x7ff838006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.091+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8480b78f0 con 0x7ff8481f8f80 2024-04-12T19:51:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.309+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8480d2460 con 0x7ff8481f8f80 2024-04-12T19:51:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.310+0000 7ff83ea00640 1 -- 172.21.15.32:0/3369432180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7ff830059a90 con 0x7ff8481f8f80 2024-04-12T19:51:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-12T19:51:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 msgr2=0x7ff82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 0x7ff82406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff838006a90 tx=0x7ff838006a20 comp rx=0 tx=0).stop 2024-04-12T19:51:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 msgr2=0x7ff8481f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff83000d8b0 tx=0x7ff83000dd70 comp rx=0 tx=0).stop 2024-04-12T19:51:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 shutdown_connections 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff82406c8f0 0x7ff82406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff84815cf60 0x7ff84815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8481f8f80 0x7ff8481f88b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 --2- 172.21.15.32:0/3369432180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff82c004d40 0x7ff8481f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 >> 172.21.15.32:0/3369432180 conn(0x7ff848073240 msgr2=0x7ff8480d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 shutdown_connections 2024-04-12T19:51:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.313+0000 7ff847e00640 1 -- 172.21.15.32:0/3369432180 wait complete. 2024-04-12T19:51:19.332 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-12T19:51:19.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:19.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-12T19:51:19.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:51:19.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:19.440 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-12T19:51:19.491 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-12T19:51:19.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.717+0000 7f2259600640 1 Processor -- start 2024-04-12T19:51:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.717+0000 7f2259600640 1 -- start start 2024-04-12T19:51:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2259600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225415b480 0x7f22540cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2259600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f225415b860 0x7f22540b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2259600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225415f2b0 con 0x7f22540cdde0 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2259600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f225415f060 con 0x7f225415b860 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2259600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f225415fec0 con 0x7f225415b480 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2258c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f225415b860 0x7f22540b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2258c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f225415b860 0x7f22540b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58740/0 (socket says 172.21.15.32:58740) 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.719+0000 7f2258c00640 1 -- 172.21.15.32:0/1979071416 learned_addr learned my addr 172.21.15.32:0/1979071416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.720+0000 7f2253e00640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225415b480 0x7f22540cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.720+0000 7f2258c00640 1 -- 172.21.15.32:0/1979071416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225415b480 msgr2=0x7f22540cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.720+0000 7f2258c00640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225415b480 0x7f22540cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.720+0000 7f2258c00640 1 -- 172.21.15.32:0/1979071416 >> v1:172.21.15.169:6789/0 conn(0x7f22540cdde0 legacy=0x7f225406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.720+0000 7f2258c00640 1 -- 172.21.15.32:0/1979071416 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225415f590 con 0x7f225415b860 2024-04-12T19:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.721+0000 7f2258c00640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f22540b3fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f225415f1e0 tx=0x7f224c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc51cf9bcce5a574 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.721+0000 7f2252a00640 1 -- 172.21.15.32:0/1979071416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c02ca50 con 0x7f225415b860 2024-04-12T19:51:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.721+0000 7f2252a00640 1 -- 172.21.15.32:0/1979071416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c02cbf0 con 0x7f225415b860 2024-04-12T19:51:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.721+0000 7f2252a00640 1 -- 172.21.15.32:0/1979071416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0343f0 con 0x7f225415b860 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 -- 172.21.15.32:0/1979071416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 msgr2=0x7f22540b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f22540b3fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f225415f1e0 tx=0x7f224c02beb0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 -- 172.21.15.32:0/1979071416 shutdown_connections 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f22540b3fc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 --2- 172.21.15.32:0/1979071416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225415b480 0x7f22540cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 -- 172.21.15.32:0/1979071416 >> 172.21.15.32:0/1979071416 conn(0x7f225406b930 msgr2=0x7f225406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 -- 172.21.15.32:0/1979071416 shutdown_connections 2024-04-12T19:51:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.722+0000 7f2259600640 1 -- 172.21.15.32:0/1979071416 wait complete. 2024-04-12T19:51:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.724+0000 7f2259600640 1 Processor -- start 2024-04-12T19:51:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.724+0000 7f2259600640 1 -- start start 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22540cdde0 0x7f225414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225415b480 0x7f2254159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22540cd550 con 0x7f225415b860 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f225415fb50 con 0x7f225415b480 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2259600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f225406f4f0 con 0x7f22540cdde0 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.725+0000 7f2258c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58754/0 (socket says 172.21.15.32:58754) 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 -- 172.21.15.32:0/907204453 learned_addr learned my addr 172.21.15.32:0/907204453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 -- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22540cdde0 msgr2=0x7f225414ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2253400640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225415b480 0x7f2254159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22540cdde0 0x7f225414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 -- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225415b480 msgr2=0x7f2254159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225415b480 0x7f2254159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 -- 172.21.15.32:0/907204453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22540c9de0 con 0x7f225415b860 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.726+0000 7f2258c00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f224c009f40 tx=0x7f224c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.727+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c043020 con 0x7f225415b860 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.727+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c03e450 con 0x7f225415b860 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.727+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22540ca0c0 con 0x7f225415b860 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.727+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c009510 con 0x7f225415b860 2024-04-12T19:51:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.727+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22540ca680 con 0x7f225415b860 2024-04-12T19:51:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.728+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22540755c0 con 0x7f225415b860 2024-04-12T19:51:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.729+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f224c046020 con 0x7f225415b860 2024-04-12T19:51:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.730+0000 7f224be00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 0x7f223406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.730+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f224c0ae440 con 0x7f225415b860 2024-04-12T19:51:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.731+0000 7f2253e00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 0x7f223406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.731+0000 7f2253e00640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 0x7f223406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22440098f0 tx=0x7f2244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.735+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f224c04d030 con 0x7f225415b860 2024-04-12T19:51:19.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.981+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2254155a30 con 0x7f225415b860 2024-04-12T19:51:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.983+0000 7f224be00640 1 -- 172.21.15.32:0/907204453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f224c07c7b0 con 0x7f225415b860 2024-04-12T19:51:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:19.983 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:51:19.983 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 msgr2=0x7f223406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 0x7f223406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22440098f0 tx=0x7f2244008040 comp rx=0 tx=0).stop 2024-04-12T19:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 msgr2=0x7f225415a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f224c009f40 tx=0x7f224c004a40 comp rx=0 tx=0).stop 2024-04-12T19:51:19.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 shutdown_connections 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f223406c8f0 0x7f223406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225415b860 0x7f225415a000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225415b480 0x7f2254159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 --2- 172.21.15.32:0/907204453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22540cdde0 0x7f225414ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 >> 172.21.15.32:0/907204453 conn(0x7f225406b930 msgr2=0x7f2254154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 shutdown_connections 2024-04-12T19:51:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:19.986+0000 7f2259600640 1 -- 172.21.15.32:0/907204453 wait complete. 2024-04-12T19:51:20.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:20.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.230+0000 7fae2e600640 1 Processor -- start 2024-04-12T19:51:20.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.230+0000 7fae2e600640 1 -- start start 2024-04-12T19:51:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.231+0000 7fae2e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae280b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.231+0000 7fae2e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.231+0000 7fae2e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae280cff90 con 0x7fae2815edd0 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.231+0000 7fae2e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fae280b6bb0 con 0x7fae280cf260 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.231+0000 7fae2e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fae280d0270 con 0x7fae280b6210 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.232+0000 7fae2dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.232+0000 7fae27e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae280b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.232+0000 7fae2dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:48092/0 (socket says 172.21.15.32:48092) 2024-04-12T19:51:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.232+0000 7fae2dc00640 1 -- 172.21.15.32:0/2132032819 learned_addr learned my addr 172.21.15.32:0/2132032819 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.233+0000 7fae27e00640 1 -- 172.21.15.32:0/2132032819 >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 msgr2=0x7fae280cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.233+0000 7fae27e00640 1 --2- 172.21.15.32:0/2132032819 >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.233+0000 7fae27e00640 1 -- 172.21.15.32:0/2132032819 >> v1:172.21.15.169:6789/0 conn(0x7fae2815edd0 legacy=0x7fae280d3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.233+0000 7fae27e00640 1 -- 172.21.15.32:0/2132032819 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae280d2d70 con 0x7fae280b6210 2024-04-12T19:51:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.233+0000 7fae2dc00640 1 --2- 172.21.15.32:0/2132032819 >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.234+0000 7fae27e00640 1 --2- 172.21.15.32:0/2132032819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae280b6670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae100097a0 tx=0x7fae1002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ed6c1f14f8ec5b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.235+0000 7fae27400640 1 -- 172.21.15.32:0/2132032819 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10031780 con 0x7fae280b6210 2024-04-12T19:51:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.235+0000 7fae27400640 1 -- 172.21.15.32:0/2132032819 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae10037070 con 0x7fae280b6210 2024-04-12T19:51:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.235+0000 7fae27400640 1 -- 172.21.15.32:0/2132032819 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10031b40 con 0x7fae280b6210 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.235+0000 7fae2e600640 1 -- 172.21.15.32:0/2132032819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 msgr2=0x7fae280b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.235+0000 7fae2e600640 1 --2- 172.21.15.32:0/2132032819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae280b6670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae100097a0 tx=0x7fae1002b820 comp rx=0 tx=0).stop 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 -- 172.21.15.32:0/2132032819 shutdown_connections 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 --2- 172.21.15.32:0/2132032819 >> v2:172.21.15.169:3300/0 conn(0x7fae280cf260 0x7fae280cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 --2- 172.21.15.32:0/2132032819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae280b6670 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 -- 172.21.15.32:0/2132032819 >> 172.21.15.32:0/2132032819 conn(0x7fae280d1210 msgr2=0x7fae280d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 -- 172.21.15.32:0/2132032819 shutdown_connections 2024-04-12T19:51:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.236+0000 7fae2e600640 1 -- 172.21.15.32:0/2132032819 wait complete. 2024-04-12T19:51:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.238+0000 7fae2e600640 1 Processor -- start 2024-04-12T19:51:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.238+0000 7fae2e600640 1 -- start start 2024-04-12T19:51:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae2814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae280cf260 0x7fae280cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae280cfc30 con 0x7fae2815edd0 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fae280d3330 con 0x7fae280cf260 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fae280b7ca0 con 0x7fae280b6210 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58762/0 (socket says 172.21.15.32:58762) 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2dc00640 1 -- 172.21.15.32:0/3381914675 learned_addr learned my addr 172.21.15.32:0/3381914675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.239+0000 7fae2d200640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae2814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae27e00640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae280cf260 0x7fae280cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 -- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 msgr2=0x7fae2814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae2814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 -- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae280cf260 msgr2=0x7fae280cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae280cf260 0x7fae280cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 -- 172.21.15.32:0/3381914675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae280ce6e0 con 0x7fae2815edd0 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae2dc00640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae2000efc0 tx=0x7fae2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000cbe0 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2000cd80 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.240+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20003bb0 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.241+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae280cc210 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.241+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae280cc7d0 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.242+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae280755c0 con 0x7fae2815edd0 2024-04-12T19:51:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.243+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae2001b030 con 0x7fae2815edd0 2024-04-12T19:51:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.244+0000 7fae24c00640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 0x7fae0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.244+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fae2008bf90 con 0x7fae2815edd0 2024-04-12T19:51:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.244+0000 7fae2d200640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 0x7fae0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.245+0000 7fae2d200640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 0x7fae0406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae18007fd0 tx=0x7fae18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.248+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae20022020 con 0x7fae2815edd0 2024-04-12T19:51:20.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.455+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae280d2760 con 0x7fae2815edd0 2024-04-12T19:51:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.457+0000 7fae24c00640 1 -- 172.21.15.32:0/3381914675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7fae20010040 con 0x7fae2815edd0 2024-04-12T19:51:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:20.458 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 msgr2=0x7fae0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 0x7fae0406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae18007fd0 tx=0x7fae18009040 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 msgr2=0x7fae280b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae2000efc0 tx=0x7fae2000c490 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 shutdown_connections 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fae0406c8f0 0x7fae0406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae2815edd0 0x7fae280b55c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae280cf260 0x7fae280cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 --2- 172.21.15.32:0/3381914675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae280b6210 0x7fae2814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.460+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 >> 172.21.15.32:0/3381914675 conn(0x7fae280d1210 msgr2=0x7fae280d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.461+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 shutdown_connections 2024-04-12T19:51:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.461+0000 7fae2e600640 1 -- 172.21.15.32:0/3381914675 wait complete. 2024-04-12T19:51:20.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:50:21.677901+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:50:23.115942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:20.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.710+0000 7fc1bf000640 1 Processor -- start 2024-04-12T19:51:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.710+0000 7fc1bf000640 1 -- start start 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc1b8145070 0x7fc1b8072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b8072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1b815f1a0 0x7fc1b815f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b81627f0 con 0x7fc1b815f1a0 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b815ffe0 con 0x7fc1b8145070 2024-04-12T19:51:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b8162a40 con 0x7fc1b8163520 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1b815f1a0 0x7fc1b815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1b815f1a0 0x7fc1b815f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58770/0 (socket says 172.21.15.32:58770) 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1bd000640 1 -- 172.21.15.32:0/2172592292 learned_addr learned my addr 172.21.15.32:0/2172592292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.712+0000 7fc1b7e00640 1 --2- 172.21.15.32:0/2172592292 >> v2:172.21.15.169:3300/0 conn(0x7fc1b8145070 0x7fc1b8072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1b7400640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b8072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1bd000640 1 -- 172.21.15.32:0/2172592292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 msgr2=0x7fc1b8072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:20.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1bd000640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b8072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1bd000640 1 -- 172.21.15.32:0/2172592292 >> v2:172.21.15.169:3300/0 conn(0x7fc1b8145070 msgr2=0x7fc1b8072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1bd000640 1 --2- 172.21.15.32:0/2172592292 >> v2:172.21.15.169:3300/0 conn(0x7fc1b8145070 0x7fc1b8072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.713+0000 7fc1bd000640 1 -- 172.21.15.32:0/2172592292 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b8162d20 con 0x7fc1b815f1a0 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.714+0000 7fc1bd000640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b815f1a0 0x7fc1b815f630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8072e80 tx=0x7fc1b002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b2482b8c79d42d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.714+0000 7fc1b6a00640 1 -- 172.21.15.32:0/2172592292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b002c9e0 con 0x7fc1b815f1a0 2024-04-12T19:51:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.714+0000 7fc1b6a00640 1 -- 172.21.15.32:0/2172592292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b002cb80 con 0x7fc1b815f1a0 2024-04-12T19:51:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.714+0000 7fc1b6a00640 1 -- 172.21.15.32:0/2172592292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00343f0 con 0x7fc1b815f1a0 2024-04-12T19:51:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 -- 172.21.15.32:0/2172592292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b815f1a0 msgr2=0x7fc1b815f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b815f1a0 0x7fc1b815f630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8072e80 tx=0x7fc1b002be40 comp rx=0 tx=0).stop 2024-04-12T19:51:20.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 -- 172.21.15.32:0/2172592292 shutdown_connections 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b815f1a0 0x7fc1b815f630 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 --2- 172.21.15.32:0/2172592292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b8072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 --2- 172.21.15.32:0/2172592292 >> v2:172.21.15.169:3300/0 conn(0x7fc1b8145070 0x7fc1b8072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 -- 172.21.15.32:0/2172592292 >> 172.21.15.32:0/2172592292 conn(0x7fc1b80b2460 msgr2=0x7fc1b80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 -- 172.21.15.32:0/2172592292 shutdown_connections 2024-04-12T19:51:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.715+0000 7fc1bf000640 1 -- 172.21.15.32:0/2172592292 wait complete. 2024-04-12T19:51:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.717+0000 7fc1bf000640 1 Processor -- start 2024-04-12T19:51:20.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.717+0000 7fc1bf000640 1 -- start start 2024-04-12T19:51:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 0x7fc1b813a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b815fb70 con 0x7fc1b8145070 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8145450 con 0x7fc1b815f1a0 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.718+0000 7fc1bf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b815fcf0 con 0x7fc1b8163520 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58780/0 (socket says 172.21.15.32:58780) 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 -- 172.21.15.32:0/513361323 learned_addr learned my addr 172.21.15.32:0/513361323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7400640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 0x7fc1b813a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 -- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 msgr2=0x7fc1b80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 -- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 msgr2=0x7fc1b813a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 0x7fc1b813a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7e00640 1 -- 172.21.15.32:0/513361323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b813bd60 con 0x7fc1b8145070 2024-04-12T19:51:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.719+0000 7fc1b7400640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 0x7fc1b813a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1b7e00640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800d800 tx=0x7fc1a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8018020 con 0x7fc1b8145070 2024-04-12T19:51:20.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a80096a0 con 0x7fc1b8145070 2024-04-12T19:51:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8009a70 con 0x7fc1b8145070 2024-04-12T19:51:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b8151220 con 0x7fc1b8145070 2024-04-12T19:51:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.720+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b813c2e0 con 0x7fc1b8145070 2024-04-12T19:51:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.721+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b8068c80 con 0x7fc1b8145070 2024-04-12T19:51:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.722+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1a801b020 con 0x7fc1b8145070 2024-04-12T19:51:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.723+0000 7fc1afe00640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 0x7fc19806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.723+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fc1a808bdd0 con 0x7fc1b8145070 2024-04-12T19:51:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.724+0000 7fc1b7400640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 0x7fc19806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.724+0000 7fc1b7400640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 0x7fc19806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.728+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1a8022030 con 0x7fc1b8145070 2024-04-12T19:51:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.931+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1b80b3d10 con 0x7fc1b8145070 2024-04-12T19:51:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.932+0000 7fc1afe00640 1 -- 172.21.15.32:0/513361323 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc1a805a140 con 0x7fc1b8145070 2024-04-12T19:51:20.932 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 msgr2=0x7fc19806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 0x7fc19806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4008040 comp rx=0 tx=0).stop 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 msgr2=0x7fc1b813cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800d800 tx=0x7fc1a800dcc0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 shutdown_connections 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc19806c920 0x7fc19806ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1b8163520 0x7fc1b80cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f1a0 0x7fc1b813a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 --2- 172.21.15.32:0/513361323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1b8145070 0x7fc1b813cae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 >> 172.21.15.32:0/513361323 conn(0x7fc1b80b2460 msgr2=0x7fc1b80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 shutdown_connections 2024-04-12T19:51:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:20.935+0000 7fc1bf000640 1 -- 172.21.15.32:0/513361323 wait complete. 2024-04-12T19:51:20.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.181+0000 7feccbe00640 1 Processor -- start 2024-04-12T19:51:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.182+0000 7feccbe00640 1 -- start start 2024-04-12T19:51:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecc415a440 0x7fecc415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fecc406fc80 0x7fecc40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc40cd620 0x7fecc4072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fecc406e280 con 0x7fecc415a440 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fecc415d520 con 0x7fecc406fc80 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.183+0000 7feccbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fecc415d800 con 0x7fecc40cd620 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecc415a440 0x7fecc415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecc415a440 0x7fecc415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58788/0 (socket says 172.21.15.32:58788) 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 -- 172.21.15.32:0/2122489711 learned_addr learned my addr 172.21.15.32:0/2122489711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc8a00640 1 --2- 172.21.15.32:0/2122489711 >> v2:172.21.15.169:3300/0 conn(0x7fecc406fc80 0x7fecc40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9e00640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc40cd620 0x7fecc4072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 -- 172.21.15.32:0/2122489711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc40cd620 msgr2=0x7fecc4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc40cd620 0x7fecc4072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 -- 172.21.15.32:0/2122489711 >> v2:172.21.15.169:3300/0 conn(0x7fecc406fc80 msgr2=0x7fecc40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 --2- 172.21.15.32:0/2122489711 >> v2:172.21.15.169:3300/0 conn(0x7fecc406fc80 0x7fecc40cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.184+0000 7fecc9400640 1 -- 172.21.15.32:0/2122489711 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc415dae0 con 0x7fecc415a440 2024-04-12T19:51:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.186+0000 7fecc9400640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc415cef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fecc40ccb70 tx=0x7fecb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca9d325d0273fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.186+0000 7fecc3e00640 1 -- 172.21.15.32:0/2122489711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8035020 con 0x7fecc415a440 2024-04-12T19:51:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.186+0000 7fecc3e00640 1 -- 172.21.15.32:0/2122489711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8031870 con 0x7fecc415a440 2024-04-12T19:51:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.186+0000 7fecc3e00640 1 -- 172.21.15.32:0/2122489711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8031c40 con 0x7fecc415a440 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 -- 172.21.15.32:0/2122489711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 msgr2=0x7fecc415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc415cef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fecc40ccb70 tx=0x7fecb802b910 comp rx=0 tx=0).stop 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 -- 172.21.15.32:0/2122489711 shutdown_connections 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc40cd620 0x7fecc4072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 --2- 172.21.15.32:0/2122489711 >> v2:172.21.15.169:3300/0 conn(0x7fecc406fc80 0x7fecc40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 --2- 172.21.15.32:0/2122489711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc415cef0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 -- 172.21.15.32:0/2122489711 >> 172.21.15.32:0/2122489711 conn(0x7fecc40b2460 msgr2=0x7fecc40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 -- 172.21.15.32:0/2122489711 shutdown_connections 2024-04-12T19:51:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.187+0000 7feccbe00640 1 -- 172.21.15.32:0/2122489711 wait complete. 2024-04-12T19:51:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.189+0000 7feccbe00640 1 Processor -- start 2024-04-12T19:51:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.189+0000 7feccbe00640 1 -- start start 2024-04-12T19:51:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc406fc80 0x7fecc413bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc40cd620 0x7fecc413c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fecc406f510 con 0x7fecc415a440 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fecc415ddc0 con 0x7fecc40cd620 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7feccbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fecc4072690 con 0x7fecc406fc80 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7fecc9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7fecc9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58790/0 (socket says 172.21.15.32:58790) 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7fecc9e00640 1 -- 172.21.15.32:0/3535962197 learned_addr learned my addr 172.21.15.32:0/3535962197 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7fecc8a00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc40cd620 0x7fecc413c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.190+0000 7fecc9400640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc406fc80 0x7fecc413bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 -- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc406fc80 msgr2=0x7fecc413bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc406fc80 0x7fecc413bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 -- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc40cd620 msgr2=0x7fecc413c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc40cd620 0x7fecc413c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 -- 172.21.15.32:0/3535962197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc415aa80 con 0x7fecc415a440 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc9e00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007e60 tx=0x7fecbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc014620 con 0x7fecc415a440 2024-04-12T19:51:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc019070 con 0x7fecc415a440 2024-04-12T19:51:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc014a00 con 0x7fecc415a440 2024-04-12T19:51:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.191+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc4153070 con 0x7fecc415a440 2024-04-12T19:51:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.192+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc4153630 con 0x7fecc415a440 2024-04-12T19:51:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.193+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc4068c80 con 0x7fecc415a440 2024-04-12T19:51:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.194+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecbc014ba0 con 0x7fecc415a440 2024-04-12T19:51:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.195+0000 7fecc1600640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.195+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fecbc08c5d0 con 0x7fecc415a440 2024-04-12T19:51:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.195+0000 7fecc9400640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.196+0000 7fecc9400640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 0x7feca406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecb8036000 tx=0x7fecb803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.199+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecbc011260 con 0x7fecc415a440 2024-04-12T19:51:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.403+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fecc415ac50 con 0x7fecc415a440 2024-04-12T19:51:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.404+0000 7fecc1600640 1 -- 172.21.15.32:0/3535962197 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fecbc05a8b0 con 0x7fecc415a440 2024-04-12T19:51:21.404 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:21.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.406+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 msgr2=0x7feca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.406+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 0x7feca406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fecb8036000 tx=0x7fecb803c040 comp rx=0 tx=0).stop 2024-04-12T19:51:21.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.406+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 msgr2=0x7fecc413c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.406+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007e60 tx=0x7fecbc00e4b0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 shutdown_connections 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7feca406c8f0 0x7feca406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecc415a440 0x7fecc413c940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc40cd620 0x7fecc413c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 --2- 172.21.15.32:0/3535962197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecc406fc80 0x7fecc413bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 >> 172.21.15.32:0/3535962197 conn(0x7fecc40b2460 msgr2=0x7fecc415f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 shutdown_connections 2024-04-12T19:51:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.407+0000 7feccbe00640 1 -- 172.21.15.32:0/3535962197 wait complete. 2024-04-12T19:51:21.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.658+0000 7f2ab7200640 1 Processor -- start 2024-04-12T19:51:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.658+0000 7f2ab7200640 1 -- start start 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2ab7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ab80b3dc0 0x7f2ab80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2ab7200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab815f230 con 0x7f2ab806f740 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2ab7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab815f4d0 con 0x7f2ab8158f10 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2ab7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab815f7b0 con 0x7f2ab80b3dc0 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2aafe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ab806f740 0x7f2ab80cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54760/0 (socket says 172.21.15.32:54760) 2024-04-12T19:51:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2aafe00640 1 -- 172.21.15.32:0/3535629209 learned_addr learned my addr 172.21.15.32:0/3535629209 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.660+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795500004 0 0) 0x7f2ab815f230 con 0x7f2ab806f740 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a9c003520 con 0x7f2ab806f740 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab6800640 1 --2- 172.21.15.32:0/3535629209 >> v2:172.21.15.169:3300/0 conn(0x7f2ab80b3dc0 0x7f2ab80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793988054 0 0) 0x7f2a9c003520 con 0x7f2ab806f740 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 >> v2:172.21.15.169:3300/0 conn(0x7f2ab80b3dc0 msgr2=0x7f2ab80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 --2- 172.21.15.32:0/3535629209 >> v2:172.21.15.169:3300/0 conn(0x7f2ab80b3dc0 0x7f2ab80cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 >> v1:172.21.15.169:6789/0 conn(0x7f2ab8158f10 legacy=0x7f2ab806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab815fa90 con 0x7f2ab806f740 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f2aa0001670 con 0x7f2ab806f740 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 >> v1:172.21.15.32:6789/0 conn(0x7f2ab806f740 legacy=0x7f2ab80cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2a9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c004800 0x7f2a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c005290 0x7f2a9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab815f7b0 con 0x7f2a9c003a00 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c00c1e0 con 0x7f2a9c004800 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.661+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c00c3e0 con 0x7f2a9c005290 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2a9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2aafe00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c004800 0x7f2a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 -- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c005290 msgr2=0x7f2a9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c005290 0x7f2a9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 -- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c004800 msgr2=0x7f2a9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c004800 0x7f2a9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab5e00640 1 -- 172.21.15.32:0/3535629209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c003520 con 0x7f2a9c003a00 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2aafe00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c004800 0x7f2a9c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:21.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.662+0000 7f2ab6800640 1 -- 172.21.15.32:0/3535629209 reap_dead start 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.663+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2a9c003e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003db0 tx=0x7f2a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebba9656a10645 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8037070 con 0x7f2a9c003a00 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa802be50 con 0x7f2a9c003a00 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab5400640 1 -- 172.21.15.32:0/3535629209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003980 con 0x7f2a9c003a00 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab7200640 1 -- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 msgr2=0x7f2a9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab7200640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2a9c003e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003db0 tx=0x7f2a9c005830 comp rx=0 tx=0).stop 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab7200640 1 -- 172.21.15.32:0/3535629209 shutdown_connections 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab7200640 1 --2- 172.21.15.32:0/3535629209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2a9c003e00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.664+0000 7f2ab7200640 1 -- 172.21.15.32:0/3535629209 >> 172.21.15.32:0/3535629209 conn(0x7f2ab806b930 msgr2=0x7f2ab806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.665+0000 7f2ab7200640 1 -- 172.21.15.32:0/3535629209 shutdown_connections 2024-04-12T19:51:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.665+0000 7f2ab7200640 1 -- 172.21.15.32:0/3535629209 wait complete. 2024-04-12T19:51:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.666+0000 7f2ab7200640 1 Processor -- start 2024-04-12T19:51:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.667+0000 7f2ab7200640 1 -- start start 2024-04-12T19:51:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ab8156500 0x7f2ab8156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 0x7f2ab8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8159470 con 0x7f2a9c003a00 2024-04-12T19:51:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81592f0 con 0x7f2ab8150a40 2024-04-12T19:51:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80cd4e0 con 0x7f2ab8156500 2024-04-12T19:51:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58814/0 (socket says 172.21.15.32:58814) 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 -- 172.21.15.32:0/1253642749 learned_addr learned my addr 172.21.15.32:0/1253642749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab6800640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 0x7f2ab8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 -- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ab8156500 msgr2=0x7f2ab8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ab8156500 0x7f2ab8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ab8156500 0x7f2ab8156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 -- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 msgr2=0x7f2ab8150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 0x7f2ab8150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab5e00640 1 -- 172.21.15.32:0/1253642749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab8156fa0 con 0x7f2a9c003a00 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.668+0000 7f2ab6800640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 0x7f2ab8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2ab5e00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003db0 tx=0x7f2aa80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8042030 con 0x7f2a9c003a00 2024-04-12T19:51:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8007470 con 0x7f2a9c003a00 2024-04-12T19:51:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8037040 con 0x7f2a9c003a00 2024-04-12T19:51:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab8157260 con 0x7f2a9c003a00 2024-04-12T19:51:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.669+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab80c9ec0 con 0x7f2a9c003a00 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.670+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab806c8c0 con 0x7f2a9c003a00 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.671+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2aa8045070 con 0x7f2a9c003a00 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.672+0000 7f2aae000640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 0x7f2a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.672+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f2aa80ad540 con 0x7f2a9c003a00 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.673+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 0x7f2a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.673+0000 7f2aafe00640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 0x7f2a9406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000fd0 tx=0x7f2aa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.677+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2aa807b8b0 con 0x7f2a9c003a00 2024-04-12T19:51:21.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.883+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ab80755c0 con 0x7f2a9c003a00 2024-04-12T19:51:21.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.884+0000 7f2aae000640 1 -- 172.21.15.32:0/1253642749 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2aa807b1b0 con 0x7f2a9c003a00 2024-04-12T19:51:21.884 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:21.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 msgr2=0x7f2a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 0x7f2a9406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000fd0 tx=0x7f2aa0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:21.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 msgr2=0x7f2ab815a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8003db0 tx=0x7f2aa80036f0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 shutdown_connections 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2a9406c8f0 0x7f2a9406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab8150a40 0x7f2ab8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ab8156500 0x7f2ab8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 --2- 172.21.15.32:0/1253642749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c003a00 0x7f2ab815a8b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 >> 172.21.15.32:0/1253642749 conn(0x7f2ab806b930 msgr2=0x7f2ab806ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 shutdown_connections 2024-04-12T19:51:21.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:21.887+0000 7f2ab7200640 1 -- 172.21.15.32:0/1253642749 wait complete. 2024-04-12T19:51:21.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.114+0000 7f845fe00640 1 Processor -- start 2024-04-12T19:51:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.115+0000 7f845fe00640 1 -- start start 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.116+0000 7f845fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84580b6e10 0x7f84580b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.116+0000 7f845fe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84580b7730 con 0x7f84580d17f0 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.116+0000 7f845fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84580b2fd0 con 0x7f84580b1390 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.116+0000 7f845fe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f84580b32b0 con 0x7f84580b6e10 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84580b6e10 0x7f84580b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845ca00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84580d17f0 0x7f84580b2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54774/0 (socket says 172.21.15.32:54774) 2024-04-12T19:51:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845ca00640 1 -- 172.21.15.32:0/3624170282 learned_addr learned my addr 172.21.15.32:0/3624170282 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845d400640 1 -- 172.21.15.32:0/3624170282 >> v1:172.21.15.169:6789/0 conn(0x7f84580b1390 legacy=0x7f84580b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845d400640 1 -- 172.21.15.32:0/3624170282 >> v1:172.21.15.32:6789/0 conn(0x7f84580d17f0 legacy=0x7f84580b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f845d400640 1 -- 172.21.15.32:0/3624170282 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84580b0b90 con 0x7f84580b6e10 2024-04-12T19:51:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.117+0000 7f8457e00640 1 -- 172.21.15.32:0/3624170282 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139117671 0 0) 0x7f84580b7730 con 0x7f84580d17f0 2024-04-12T19:51:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f845d400640 1 --2- 172.21.15.32:0/3624170282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b6e10 0x7f84580b71f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f844c0097a0 tx=0x7f844c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1788e282597a9b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f8457e00640 1 -- 172.21.15.32:0/3624170282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c034070 con 0x7f84580b6e10 2024-04-12T19:51:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f8457e00640 1 -- 172.21.15.32:0/3624170282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c030610 con 0x7f84580b6e10 2024-04-12T19:51:22.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f8457e00640 1 -- 172.21.15.32:0/3624170282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0309e0 con 0x7f84580b6e10 2024-04-12T19:51:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f845fe00640 1 -- 172.21.15.32:0/3624170282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b6e10 msgr2=0x7f84580b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.119+0000 7f845fe00640 1 --2- 172.21.15.32:0/3624170282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b6e10 0x7f84580b71f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f844c0097a0 tx=0x7f844c02b960 comp rx=0 tx=0).stop 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.120+0000 7f845fe00640 1 -- 172.21.15.32:0/3624170282 shutdown_connections 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.120+0000 7f845fe00640 1 --2- 172.21.15.32:0/3624170282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b6e10 0x7f84580b71f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.120+0000 7f845fe00640 1 -- 172.21.15.32:0/3624170282 >> 172.21.15.32:0/3624170282 conn(0x7f845809aa40 msgr2=0x7f845809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.120+0000 7f845fe00640 1 -- 172.21.15.32:0/3624170282 shutdown_connections 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.120+0000 7f845fe00640 1 -- 172.21.15.32:0/3624170282 wait complete. 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.121+0000 7f845fe00640 1 Processor -- start 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.122+0000 7f845fe00640 1 -- start start 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.122+0000 7f845fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.122+0000 7f845fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 0x7f845813d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.122+0000 7f845fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84580d17f0 0x7f845809c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8458146550 con 0x7f84580b1390 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84580b0870 con 0x7f84580b6e10 2024-04-12T19:51:22.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84581466d0 con 0x7f84580d17f0 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58832/0 (socket says 172.21.15.32:58832) 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 -- 172.21.15.32:0/2278294857 learned_addr learned my addr 172.21.15.32:0/2278294857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845ca00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 0x7f845813d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 -- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84580d17f0 msgr2=0x7f845809c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84580d17f0 0x7f845809c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 -- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 msgr2=0x7f845813d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 0x7f845813d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845d400640 1 -- 172.21.15.32:0/2278294857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458154870 con 0x7f84580b1390 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.123+0000 7f845ca00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 0x7f845813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f845d400640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f844c0098c0 tx=0x7f844c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c034040 con 0x7f84580b1390 2024-04-12T19:51:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c039a10 con 0x7f84580b1390 2024-04-12T19:51:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c039de0 con 0x7f84580b1390 2024-04-12T19:51:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845809bbf0 con 0x7f84580b1390 2024-04-12T19:51:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.124+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845809c1b0 con 0x7f84580b1390 2024-04-12T19:51:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.125+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8458068c80 con 0x7f84580b1390 2024-04-12T19:51:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.127+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f844c037070 con 0x7f84580b1390 2024-04-12T19:51:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.128+0000 7f8455600640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.128+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f844c0add80 con 0x7f84580b1390 2024-04-12T19:51:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.128+0000 7f845ca00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.129+0000 7f845ca00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f845815fe20 tx=0x7f8448009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.133+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f844c07c0f0 con 0x7f84580b1390 2024-04-12T19:51:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.381+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84580d1e80 con 0x7f84580b1390 2024-04-12T19:51:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.382+0000 7f8455600640 1 -- 172.21.15.32:0/2278294857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f844c07b9f0 con 0x7f84580b1390 2024-04-12T19:51:22.383 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:22.383 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:51:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 msgr2=0x7f843c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f845815fe20 tx=0x7f8448009040 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 msgr2=0x7f8458145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f844c0098c0 tx=0x7f844c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 shutdown_connections 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84580d17f0 0x7f845809c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84580b6e10 0x7f845813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 --2- 172.21.15.32:0/2278294857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84580b1390 0x7f8458145800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 >> 172.21.15.32:0/2278294857 conn(0x7f845809aa40 msgr2=0x7f845806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.385+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 shutdown_connections 2024-04-12T19:51:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.386+0000 7f845fe00640 1 -- 172.21.15.32:0/2278294857 wait complete. 2024-04-12T19:51:22.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.618+0000 7ff11e000640 1 Processor -- start 2024-04-12T19:51:22.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.618+0000 7ff11e000640 1 -- start start 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff118072660 0x7ff11806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11e000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11815dbb0 con 0x7ff11815cbe0 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11815d8d0 con 0x7ff1180cee00 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff11806f4f0 con 0x7ff118072660 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11d600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff11815cbe0 0x7ff11815d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54786/0 (socket says 172.21.15.32:54786) 2024-04-12T19:51:22.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.620+0000 7ff11d600640 1 -- 172.21.15.32:0/4083669490 learned_addr learned my addr 172.21.15.32:0/4083669490 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117e00640 1 --2- 172.21.15.32:0/4083669490 >> v2:172.21.15.169:3300/0 conn(0x7ff118072660 0x7ff11806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334940140 0 0) 0x7ff11815dbb0 con 0x7ff11815cbe0 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff100003520 con 0x7ff11815cbe0 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 655786320 0 0) 0x7ff100003520 con 0x7ff11815cbe0 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 >> v2:172.21.15.169:3300/0 conn(0x7ff118072660 msgr2=0x7ff11806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 --2- 172.21.15.32:0/4083669490 >> v2:172.21.15.169:3300/0 conn(0x7ff118072660 0x7ff11806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 >> v1:172.21.15.169:6789/0 conn(0x7ff1180cee00 legacy=0x7ff118071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11806dde0 con 0x7ff11815cbe0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7ff1100019f0 con 0x7ff11815cbe0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 >> v1:172.21.15.32:6789/0 conn(0x7ff11815cbe0 legacy=0x7ff11815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000039b0 0x7ff100003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000051c0 0x7ff10000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff10000c0e0 con 0x7ff100004800 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff10000c290 con 0x7ff1000039b0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff10000c490 con 0x7ff1000051c0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.621+0000 7ff117e00640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff11cc00640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000039b0 0x7ff100003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff117e00640 1 -- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000051c0 msgr2=0x7ff10000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff117e00640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1000051c0 0x7ff10000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff117e00640 1 -- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000039b0 msgr2=0x7ff100003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff117e00640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1000039b0 0x7ff100003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff117e00640 1 -- 172.21.15.32:0/4083669490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff100003520 con 0x7ff100004800 2024-04-12T19:51:22.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.622+0000 7ff11d600640 1 -- 172.21.15.32:0/4083669490 reap_dead start 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.623+0000 7ff117e00640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff100004c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff10400ba30 tx=0x7ff100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12163be21b124988 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.623+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104036020 con 0x7ff100004800 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.623+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff104032640 con 0x7ff100004800 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.623+0000 7ff117400640 1 -- 172.21.15.32:0/4083669490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400b600 con 0x7ff100004800 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 -- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 msgr2=0x7ff100004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff100004c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff10400ba30 tx=0x7ff100005830 comp rx=0 tx=0).stop 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 -- 172.21.15.32:0/4083669490 shutdown_connections 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 --2- 172.21.15.32:0/4083669490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff100004c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 -- 172.21.15.32:0/4083669490 >> 172.21.15.32:0/4083669490 conn(0x7ff11806b930 msgr2=0x7ff11806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 -- 172.21.15.32:0/4083669490 shutdown_connections 2024-04-12T19:51:22.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.624+0000 7ff11e000640 1 -- 172.21.15.32:0/4083669490 wait complete. 2024-04-12T19:51:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.626+0000 7ff11e000640 1 Processor -- start 2024-04-12T19:51:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.626+0000 7ff11e000640 1 -- start start 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 0x7ff118153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 0x7ff118158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806e8c0 con 0x7ff100004800 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806e3a0 con 0x7ff118153a50 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff11806ea40 con 0x7ff118158010 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58846/0 (socket says 172.21.15.32:58846) 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11cc00640 1 -- 172.21.15.32:0/993557046 learned_addr learned my addr 172.21.15.32:0/993557046 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff117e00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 0x7ff118153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11d600640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 0x7ff118158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.627+0000 7ff11cc00640 1 -- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 msgr2=0x7ff118158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11cc00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 0x7ff118158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11cc00640 1 -- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 msgr2=0x7ff118153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11cc00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 0x7ff118153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11cc00640 1 -- 172.21.15.32:0/993557046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11814fa80 con 0x7ff100004800 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff117e00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 0x7ff118153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11d600640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 0x7ff118158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:22.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11cc00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00d4c0 tx=0x7ff10c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005480 con 0x7ff100004800 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10c017070 con 0x7ff100004800 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c003830 con 0x7ff100004800 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11814fd60 con 0x7ff100004800 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.628+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff118150320 con 0x7ff100004800 2024-04-12T19:51:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.629+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1180755c0 con 0x7ff100004800 2024-04-12T19:51:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.630+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff10c0039d0 con 0x7ff100004800 2024-04-12T19:51:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.631+0000 7ff114c00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 0x7ff0f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.631+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7ff10c08ba50 con 0x7ff100004800 2024-04-12T19:51:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.631+0000 7ff117e00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 0x7ff0f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.632+0000 7ff117e00640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 0x7ff0f806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff10400ba00 tx=0x7ff10403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.636+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff10c059d30 con 0x7ff100004800 2024-04-12T19:51:22.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.842+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff118156c70 con 0x7ff100004800 2024-04-12T19:51:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.844+0000 7ff114c00640 1 -- 172.21.15.32:0/993557046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7ff10c059630 con 0x7ff100004800 2024-04-12T19:51:22.845 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:22.845 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.847+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 msgr2=0x7ff0f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.847+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 0x7ff0f806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff10400ba00 tx=0x7ff10403d040 comp rx=0 tx=0).stop 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.847+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 msgr2=0x7ff11814f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00d4c0 tx=0x7ff10c00d980 comp rx=0 tx=0).stop 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 shutdown_connections 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff0f806c920 0x7ff0f806ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff118158010 0x7ff118158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118153a50 0x7ff118153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 --2- 172.21.15.32:0/993557046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff100004800 0x7ff11814f010 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 >> 172.21.15.32:0/993557046 conn(0x7ff11806b930 msgr2=0x7ff1180b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 shutdown_connections 2024-04-12T19:51:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:22.848+0000 7ff11e000640 1 -- 172.21.15.32:0/993557046 wait complete. 2024-04-12T19:51:22.868 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-12T19:51:22.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.089+0000 7f9cb8800640 1 Processor -- start 2024-04-12T19:51:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.089+0000 7f9cb8800640 1 -- start start 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb8800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb015b6d0 0x7f9cb00cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb0070210 0x7f9cb00b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb8800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb006f940 con 0x7f9cb00ce250 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb8800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb006e8a0 con 0x7f9cb015b6d0 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb006eb80 con 0x7f9cb0070210 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb5e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb015b6d0 0x7f9cb00cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb5e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cb015b6d0 0x7f9cb00cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58852/0 (socket says 172.21.15.32:58852) 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.091+0000 7f9cb5e00640 1 -- 172.21.15.32:0/4004072308 learned_addr learned my addr 172.21.15.32:0/4004072308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.092+0000 7f9cb5400640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb0070210 0x7f9cb00b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.092+0000 7f9cb5e00640 1 -- 172.21.15.32:0/4004072308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb0070210 msgr2=0x7f9cb00b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.092+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb0070210 0x7f9cb00b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.092+0000 7f9cb5e00640 1 -- 172.21.15.32:0/4004072308 >> v1:172.21.15.169:6789/0 conn(0x7f9cb00ce250 legacy=0x7f9cb00721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.092+0000 7f9cb5e00640 1 -- 172.21.15.32:0/4004072308 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb006ee60 con 0x7f9cb015b6d0 2024-04-12T19:51:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.093+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb015b6d0 0x7f9cb00cdd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009870 tx=0x7f9c9c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1c6eac18da9fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.093+0000 7f9cb4a00640 1 -- 172.21.15.32:0/4004072308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0317c0 con 0x7f9cb015b6d0 2024-04-12T19:51:23.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.093+0000 7f9cb4a00640 1 -- 172.21.15.32:0/4004072308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c037070 con 0x7f9cb015b6d0 2024-04-12T19:51:23.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.093+0000 7f9cb4a00640 1 -- 172.21.15.32:0/4004072308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c031b80 con 0x7f9cb015b6d0 2024-04-12T19:51:23.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 -- 172.21.15.32:0/4004072308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb015b6d0 msgr2=0x7f9cb00cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb015b6d0 0x7f9cb00cdd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009870 tx=0x7f9c9c02b860 comp rx=0 tx=0).stop 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 -- 172.21.15.32:0/4004072308 shutdown_connections 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb0070210 0x7f9cb00b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 --2- 172.21.15.32:0/4004072308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb015b6d0 0x7f9cb00cdd10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 -- 172.21.15.32:0/4004072308 >> 172.21.15.32:0/4004072308 conn(0x7f9cb00b27b0 msgr2=0x7f9cb00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 -- 172.21.15.32:0/4004072308 shutdown_connections 2024-04-12T19:51:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.094+0000 7f9cb8800640 1 -- 172.21.15.32:0/4004072308 wait complete. 2024-04-12T19:51:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.096+0000 7f9cb8800640 1 Processor -- start 2024-04-12T19:51:23.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.096+0000 7f9cb8800640 1 -- start start 2024-04-12T19:51:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 0x7f9cb014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb015b6d0 0x7f9cb00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb015cac0 con 0x7f9cb00ce250 2024-04-12T19:51:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb006f420 con 0x7f9cb0070210 2024-04-12T19:51:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb015cc40 con 0x7f9cb015b6d0 2024-04-12T19:51:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58854/0 (socket says 172.21.15.32:58854) 2024-04-12T19:51:23.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.097+0000 7f9cb5400640 1 -- 172.21.15.32:0/3585418215 learned_addr learned my addr 172.21.15.32:0/3585418215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 0x7f9cb014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb6800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb015b6d0 0x7f9cb00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 -- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb015b6d0 msgr2=0x7f9cb00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb015b6d0 0x7f9cb00c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 -- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 msgr2=0x7f9cb014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 0x7f9cb014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 -- 172.21.15.32:0/3585418215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00ae580 con 0x7f9cb00ce250 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 0x7f9cb014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9cb5400640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c7f0 tx=0x7f9ca400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4015070 con 0x7f9cb00ce250 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4004240 con 0x7f9cb00ce250 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.098+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4007620 con 0x7f9cb00ce250 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.099+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb00ae860 con 0x7f9cb00ce250 2024-04-12T19:51:23.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.099+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb00aee20 con 0x7f9cb00ce250 2024-04-12T19:51:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.100+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb0068c80 con 0x7f9cb00ce250 2024-04-12T19:51:23.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.101+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ca40079d0 con 0x7f9cb00ce250 2024-04-12T19:51:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.102+0000 7f9caa000640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 0x7f9c9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.102+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f9ca408bcd0 con 0x7f9cb00ce250 2024-04-12T19:51:23.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.102+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 0x7f9c9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.103+0000 7f9cb5e00640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 0x7f9c9006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c036000 tx=0x7f9c9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.106+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ca405a040 con 0x7f9cb00ce250 2024-04-12T19:51:23.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.349+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cb00d0560 con 0x7f9cb00ce250 2024-04-12T19:51:23.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.350+0000 7f9caa000640 1 -- 172.21.15.32:0/3585418215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f9ca4059940 con 0x7f9cb00ce250 2024-04-12T19:51:23.356 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:23.356 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:51:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:51:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 msgr2=0x7f9c9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 0x7f9c9006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c036000 tx=0x7f9c9c008030 comp rx=0 tx=0).stop 2024-04-12T19:51:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 msgr2=0x7f9cb00c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c7f0 tx=0x7f9ca400ccb0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 shutdown_connections 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9c9006c920 0x7f9c9006ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cb015b6d0 0x7f9cb00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cb00ce250 0x7f9cb00c7d40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 --2- 172.21.15.32:0/3585418215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb0070210 0x7f9cb014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 >> 172.21.15.32:0/3585418215 conn(0x7f9cb00b27b0 msgr2=0x7f9cb006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 shutdown_connections 2024-04-12T19:51:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.354+0000 7f9cb8800640 1 -- 172.21.15.32:0/3585418215 wait complete. 2024-04-12T19:51:23.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.596+0000 7f365b200640 1 Processor -- start 2024-04-12T19:51:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.596+0000 7f365b200640 1 -- start start 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c15b480 0x7f365c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f365c0cdde0 0x7f365c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 0x7f365c06f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f365c15f060 con 0x7f365c0cdde0 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f365c15fec0 con 0x7f365c15b860 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f365b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f365c15f2b0 con 0x7f365c15b480 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f3651400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f365c0cdde0 0x7f365c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f3651400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f365c0cdde0 0x7f365c06ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58862/0 (socket says 172.21.15.32:58862) 2024-04-12T19:51:23.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.598+0000 7f3651400640 1 -- 172.21.15.32:0/4158057333 learned_addr learned my addr 172.21.15.32:0/4158057333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3659e00640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c15b480 0x7f365c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f365a800640 1 --2- 172.21.15.32:0/4158057333 >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 0x7f365c06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3651400640 1 -- 172.21.15.32:0/4158057333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c15b480 msgr2=0x7f365c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3651400640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c15b480 0x7f365c0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3651400640 1 -- 172.21.15.32:0/4158057333 >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 msgr2=0x7f365c06f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3651400640 1 --2- 172.21.15.32:0/4158057333 >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 0x7f365c06f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f3651400640 1 -- 172.21.15.32:0/4158057333 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c15f590 con 0x7f365c0cdde0 2024-04-12T19:51:23.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.599+0000 7f365a800640 1 --2- 172.21.15.32:0/4158057333 >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 0x7f365c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:23.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.600+0000 7f3651400640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0cdde0 0x7f365c06ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f365c0d0570 tx=0x7f364402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1b97f8eac22ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.600+0000 7f3659400640 1 -- 172.21.15.32:0/4158057333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644035020 con 0x7f365c0cdde0 2024-04-12T19:51:23.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.600+0000 7f3659400640 1 -- 172.21.15.32:0/4158057333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644031870 con 0x7f365c0cdde0 2024-04-12T19:51:23.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.600+0000 7f3659400640 1 -- 172.21.15.32:0/4158057333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644031c40 con 0x7f365c0cdde0 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 -- 172.21.15.32:0/4158057333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0cdde0 msgr2=0x7f365c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0cdde0 0x7f365c06ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f365c0d0570 tx=0x7f364402b910 comp rx=0 tx=0).stop 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 -- 172.21.15.32:0/4158057333 shutdown_connections 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c0cdde0 0x7f365c06ede0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 --2- 172.21.15.32:0/4158057333 >> v2:172.21.15.169:3300/0 conn(0x7f365c15b860 0x7f365c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 --2- 172.21.15.32:0/4158057333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c15b480 0x7f365c0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 -- 172.21.15.32:0/4158057333 >> 172.21.15.32:0/4158057333 conn(0x7f365c06b930 msgr2=0x7f365c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 -- 172.21.15.32:0/4158057333 shutdown_connections 2024-04-12T19:51:23.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.601+0000 7f365b200640 1 -- 172.21.15.32:0/4158057333 wait complete. 2024-04-12T19:51:23.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.604+0000 7f365b200640 1 Processor -- start 2024-04-12T19:51:23.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.604+0000 7f365b200640 1 -- start start 2024-04-12T19:51:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c0cdde0 0x7f365c143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c15b480 0x7f365c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f365c0cd550 con 0x7f365c15b860 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f365c15fb50 con 0x7f365c15b480 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f365c071f50 con 0x7f365c0cdde0 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58866/0 (socket says 172.21.15.32:58866) 2024-04-12T19:51:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.605+0000 7f365a800640 1 -- 172.21.15.32:0/1684624218 learned_addr learned my addr 172.21.15.32:0/1684624218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:23.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f3659e00640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c0cdde0 0x7f365c143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f3651400640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c15b480 0x7f365c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 -- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c0cdde0 msgr2=0x7f365c143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c0cdde0 0x7f365c143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 -- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c15b480 msgr2=0x7f365c0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c15b480 0x7f365c0cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 -- 172.21.15.32:0/1684624218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c0c7860 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.606+0000 7f365a800640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f365400cd20 tx=0x7f365400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.607+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400f680 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.607+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365400f840 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.607+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400fc30 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.607+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c0c7b40 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.607+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c0c8100 con 0x7f365c15b860 2024-04-12T19:51:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.608+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c0755c0 con 0x7f365c15b860 2024-04-12T19:51:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.609+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36540213f0 con 0x7f365c15b860 2024-04-12T19:51:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.610+0000 7f3652a00640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 0x7f363806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.610+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f365408c990 con 0x7f365c15b860 2024-04-12T19:51:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.610+0000 7f3659e00640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 0x7f363806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.611+0000 7f3659e00640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 0x7f363806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f364c006fd0 tx=0x7f364c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.615+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3654014030 con 0x7f365c15b860 2024-04-12T19:51:23.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.856+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f365c1443a0 con 0x7f365c15b860 2024-04-12T19:51:23.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.857+0000 7f3652a00640 1 -- 172.21.15.32:0/1684624218 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5181 (secure 0 0 0) 0x7f365405ac70 con 0x7f365c15b860 2024-04-12T19:51:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:23.857 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3312530269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3312530269},{"type":"v1","addr":"172.21.15.169:6835","nonce":3312530269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3690950117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3690950117},{"type":"v1","addr":"172.21.15.32:6836","nonce":3690950117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3504224635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3504224635},{"type":"v1","addr":"172.21.15.32:6837","nonce":3504224635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:50:24.149610+0000","modified":"2024-04-12T19:50:27.198866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/1878337294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1878337294},{"type":"v1","addr":"172.21.15.169:6834","nonce":1878337294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4291,"qdb_cluster":[4291]},"id":1}]} 2024-04-12T19:51:23.857 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 msgr2=0x7f363806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 0x7f363806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f364c006fd0 tx=0x7f364c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 msgr2=0x7f365c1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f365400cd20 tx=0x7f365400a650 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 shutdown_connections 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f363806c8f0 0x7f363806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f365c15b860 0x7f365c1559e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c15b480 0x7f365c0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 --2- 172.21.15.32:0/1684624218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f365c0cdde0 0x7f365c143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.859+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 >> 172.21.15.32:0/1684624218 conn(0x7f365c06b930 msgr2=0x7f365c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.860+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 shutdown_connections 2024-04-12T19:51:23.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:23.860+0000 7f365b200640 1 -- 172.21.15.32:0/1684624218 wait complete. 2024-04-12T19:51:23.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.102+0000 7f150e600640 1 Processor -- start 2024-04-12T19:51:24.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.102+0000 7f150e600640 1 -- start start 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.103+0000 7f150e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f1508162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.103+0000 7f150e600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15080d1c90 0x7f15080d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f150e600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15080d0090 con 0x7f15080cf110 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f150e600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1508163440 con 0x7f15080d1c90 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f150e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15080b7b30 con 0x7f1508162020 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f1507e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15080cf110 0x7f15080d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56630/0 (socket says 172.21.15.32:56630) 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f1507e00640 1 -- 172.21.15.32:0/1770633456 learned_addr learned my addr 172.21.15.32:0/1770633456 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f150dc00640 1 --2- 172.21.15.32:0/1770633456 >> v2:172.21.15.32:3300/0 conn(0x7f15080d1c90 0x7f15080d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f150d200640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f1508162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f1507400640 1 -- 172.21.15.32:0/1770633456 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371310025 0 0) 0x7f14f4001110 con 0x7f15080cf110 2024-04-12T19:51:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.104+0000 7f1507400640 1 -- 172.21.15.32:0/1770633456 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f0003520 con 0x7f15080cf110 2024-04-12T19:51:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.105+0000 7f150dc00640 1 -- 172.21.15.32:0/1770633456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 msgr2=0x7f1508162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:24.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.105+0000 7f150dc00640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f1508162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.105+0000 7f150dc00640 1 -- 172.21.15.32:0/1770633456 >> v1:172.21.15.32:6789/0 conn(0x7f15080cf110 legacy=0x7f15080d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.105+0000 7f150dc00640 1 -- 172.21.15.32:0/1770633456 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15080cfad0 con 0x7f15080d1c90 2024-04-12T19:51:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.106+0000 7f150dc00640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f15080d3390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7ae3ce3c1b56cb server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.106+0000 7f1507400640 1 -- 172.21.15.32:0/1770633456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500039070 con 0x7f15080d1c90 2024-04-12T19:51:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.106+0000 7f1507400640 1 -- 172.21.15.32:0/1770633456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150002cc50 con 0x7f15080d1c90 2024-04-12T19:51:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.106+0000 7f1507400640 1 -- 172.21.15.32:0/1770633456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500034440 con 0x7f15080d1c90 2024-04-12T19:51:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 -- 172.21.15.32:0/1770633456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 msgr2=0x7f15080d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f15080d3390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150002bea0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 -- 172.21.15.32:0/1770633456 shutdown_connections 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f15080d3390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 --2- 172.21.15.32:0/1770633456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f1508162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 -- 172.21.15.32:0/1770633456 >> 172.21.15.32:0/1770633456 conn(0x7f150806cf00 msgr2=0x7f150806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 -- 172.21.15.32:0/1770633456 shutdown_connections 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.107+0000 7f150e600640 1 -- 172.21.15.32:0/1770633456 wait complete. 2024-04-12T19:51:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.109+0000 7f150e600640 1 Processor -- start 2024-04-12T19:51:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.109+0000 7f150e600640 1 -- start start 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 0x7f1508149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f15080d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15080d2b90 con 0x7f15080d1c90 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15080cf880 con 0x7f15080cf110 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15080d2e70 con 0x7f1508162020 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44472/0 (socket says 172.21.15.32:44472) 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 -- 172.21.15.32:0/1023390334 learned_addr learned my addr 172.21.15.32:0/1023390334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150d200640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 0x7f1508149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150dc00640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f15080d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 -- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 msgr2=0x7f15080d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f15080d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 -- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 msgr2=0x7f1508149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 0x7f1508149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f1507e00640 1 -- 172.21.15.32:0/1023390334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150815e9e0 con 0x7f15080d1c90 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.110+0000 7f150d200640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 0x7f1508149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f1507e00640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c3a0 tx=0x7f14f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017070 con 0x7f15080d1c90 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4005ce0 con 0x7f15080d1c90 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007310 con 0x7f15080d1c90 2024-04-12T19:51:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150815ecc0 con 0x7f15080d1c90 2024-04-12T19:51:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.111+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150815f280 con 0x7f15080d1c90 2024-04-12T19:51:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.112+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15080755c0 con 0x7f15080d1c90 2024-04-12T19:51:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.113+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14f401a070 con 0x7f15080d1c90 2024-04-12T19:51:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.114+0000 7f1504c00640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 0x7f14e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.115+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f14f408aff0 con 0x7f15080d1c90 2024-04-12T19:51:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.115+0000 7f150d200640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 0x7f14e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.115+0000 7f150d200640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 0x7f14e806ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.119+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15080d0090 con 0x7f15080d1c90 2024-04-12T19:51:24.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.328+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15080b7650 con 0x7f15080d1c90 2024-04-12T19:51:24.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.331+0000 7f1504c00640 1 -- 172.21.15.32:0/1023390334 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16646 (secure 0 0 0) 0x7f14f4059360 con 0x7f15080d1c90 2024-04-12T19:51:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:24.331 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:04.892371+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:50:21.677901+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-12T19:50:23.115942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.333+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 msgr2=0x7f14e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.333+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 0x7f14e806ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.333+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 msgr2=0x7f150815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.333+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c3a0 tx=0x7f14f400c860 comp rx=0 tx=0).stop 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 shutdown_connections 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f14e806c920 0x7f14e806ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1508162020 0x7f15080d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15080d1c90 0x7f150815df60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 --2- 172.21.15.32:0/1023390334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15080cf110 0x7f1508149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 >> 172.21.15.32:0/1023390334 conn(0x7f150806cf00 msgr2=0x7f15080cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 shutdown_connections 2024-04-12T19:51:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.334+0000 7f150e600640 1 -- 172.21.15.32:0/1023390334 wait complete. 2024-04-12T19:51:24.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-12T19:51:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.578+0000 7fcdb6c00640 1 Processor -- start 2024-04-12T19:51:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.578+0000 7fcdb6c00640 1 -- start start 2024-04-12T19:51:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.580+0000 7fcdb6c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdb00b5790 0x7fcdb00b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.580+0000 7fcdb6c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdb00b60b0 0x7fcdb00b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.580+0000 7fcdb6c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb00b3d00 con 0x7fcdb01625d0 2024-04-12T19:51:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.580+0000 7fcdb6c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb015efe0 con 0x7fcdb00b60b0 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.580+0000 7fcdb6c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb015f2a0 con 0x7fcdb00b5790 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdb00b60b0 0x7fcdb00b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdb00b60b0 0x7fcdb00b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44486/0 (socket says 172.21.15.32:44486) 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 -- 172.21.15.32:0/2863608565 learned_addr learned my addr 172.21.15.32:0/2863608565 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdafe00640 1 --2- 172.21.15.32:0/2863608565 >> v2:172.21.15.169:3300/0 conn(0x7fcdb00b5790 0x7fcdb00b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 -- 172.21.15.32:0/2863608565 >> v2:172.21.15.169:3300/0 conn(0x7fcdb00b5790 msgr2=0x7fcdb00b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 --2- 172.21.15.32:0/2863608565 >> v2:172.21.15.169:3300/0 conn(0x7fcdb00b5790 0x7fcdb00b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 -- 172.21.15.32:0/2863608565 >> v1:172.21.15.169:6789/0 conn(0x7fcdb01625d0 legacy=0x7fcdb00b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.581+0000 7fcdb4c00640 1 -- 172.21.15.32:0/2863608565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb015f580 con 0x7fcdb00b60b0 2024-04-12T19:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.582+0000 7fcdb4c00640 1 --2- 172.21.15.32:0/2863608565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00b4830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0068be0 tx=0x7fcda802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9482240b89bfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdaf400640 1 -- 172.21.15.32:0/2863608565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8035020 con 0x7fcdb00b60b0 2024-04-12T19:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdaf400640 1 -- 172.21.15.32:0/2863608565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda8031870 con 0x7fcdb00b60b0 2024-04-12T19:51:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdaf400640 1 -- 172.21.15.32:0/2863608565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8031c40 con 0x7fcdb00b60b0 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 -- 172.21.15.32:0/2863608565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b60b0 msgr2=0x7fcdb00b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/2863608565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00b4830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0068be0 tx=0x7fcda802b910 comp rx=0 tx=0).stop 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 -- 172.21.15.32:0/2863608565 shutdown_connections 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/2863608565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00b4830 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/2863608565 >> v2:172.21.15.169:3300/0 conn(0x7fcdb00b5790 0x7fcdb00b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.583+0000 7fcdb6c00640 1 -- 172.21.15.32:0/2863608565 >> 172.21.15.32:0/2863608565 conn(0x7fcdb006eb10 msgr2=0x7fcdb006ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.584+0000 7fcdb6c00640 1 -- 172.21.15.32:0/2863608565 shutdown_connections 2024-04-12T19:51:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.584+0000 7fcdb6c00640 1 -- 172.21.15.32:0/2863608565 wait complete. 2024-04-12T19:51:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.585+0000 7fcdb6c00640 1 Processor -- start 2024-04-12T19:51:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.586+0000 7fcdb6c00640 1 -- start start 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.586+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.586+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdb6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdb00ceac0 0x7fcdb00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb015c850 con 0x7fcdb00b5790 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb015fb40 con 0x7fcdb00b60b0 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdb6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb015c9d0 con 0x7fcdb00ceac0 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44502/0 (socket says 172.21.15.32:44502) 2024-04-12T19:51:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 -- 172.21.15.32:0/1854673869 learned_addr learned my addr 172.21.15.32:0/1854673869 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 -- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdb00ceac0 msgr2=0x7fcdb00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdb00ceac0 0x7fcdb00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 -- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00b60b0 msgr2=0x7fcdb00af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.587+0000 7fcdafe00640 1 -- 172.21.15.32:0/1854673869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb00cefc0 con 0x7fcdb00b5790 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdafe00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcda000d800 tx=0x7fcda000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda000b6c0 con 0x7fcdb00b5790 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda0019070 con 0x7fcdb00b5790 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda000ba80 con 0x7fcdb00b5790 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb00cf2a0 con 0x7fcdb00b5790 2024-04-12T19:51:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.588+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb00cf860 con 0x7fcdb00b5790 2024-04-12T19:51:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.589+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb0068c80 con 0x7fcdb00b5790 2024-04-12T19:51:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.590+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcda000bc20 con 0x7fcdb00b5790 2024-04-12T19:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.591+0000 7fcdacc00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 0x7fcd8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:24.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.591+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fcda008c100 con 0x7fcdb00b5790 2024-04-12T19:51:24.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.592+0000 7fcda7400640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 0x7fcd8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:24.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.592+0000 7fcda7400640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 0x7fcd8c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00cce00 tx=0x7fcd98009230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.596+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcda001c360 con 0x7fcdb00b5790 2024-04-12T19:51:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.839+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcdb006f940 con 0x7fcdb00b5790 2024-04-12T19:51:24.841 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-12T19:51:24.840+0000 7f5896200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T19:51:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.845+0000 7fcdacc00640 1 -- 172.21.15.32:0/1854673869 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fcda005a470 con 0x7fcdb00b5790 2024-04-12T19:51:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-12T19:51:24.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.847+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 msgr2=0x7fcd8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 0x7fcd8c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00cce00 tx=0x7fcd98009230 comp rx=0 tx=0).stop 2024-04-12T19:51:24.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 msgr2=0x7fcdb01f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:24.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcda000d800 tx=0x7fcda000dcc0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 shutdown_connections 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fcd8c06c8f0 0x7fcd8c06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdb00ceac0 0x7fcdb00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdb00b60b0 0x7fcdb00af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 --2- 172.21.15.32:0/1854673869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb00b5790 0x7fcdb01f47e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 >> 172.21.15.32:0/1854673869 conn(0x7fcdb006eb10 msgr2=0x7fcdb006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.848+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 shutdown_connections 2024-04-12T19:51:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:24.849+0000 7fcdb6c00640 1 -- 172.21.15.32:0/1854673869 wait complete. 2024-04-12T19:51:24.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-12T19:51:24.977 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-12T19:51:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.098+0000 7f63db200640 1 Processor -- start 2024-04-12T19:51:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.098+0000 7f63db200640 1 -- start start 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.099+0000 7f63db200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63dc1461c0 0x7f63dc071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.099+0000 7f63db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.099+0000 7f63db200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63dc0cd2d0 con 0x7f63dc072420 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63db200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f63dc0cf1d0 con 0x7f63dc1461c0 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc072880 con 0x7f63dc0cebb0 2024-04-12T19:51:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63d9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63dc1461c0 0x7f63dc071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:46396/0 (socket says 172.21.15.32:46396) 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.100+0000 7f63da800640 1 -- 172.21.15.32:0/982803612 learned_addr learned my addr 172.21.15.32:0/982803612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.101+0000 7f63da800640 1 -- 172.21.15.32:0/982803612 >> v2:172.21.15.169:3300/0 conn(0x7f63dc1461c0 msgr2=0x7f63dc071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.101+0000 7f63da800640 1 --2- 172.21.15.32:0/982803612 >> v2:172.21.15.169:3300/0 conn(0x7f63dc1461c0 0x7f63dc071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.101+0000 7f63da800640 1 -- 172.21.15.32:0/982803612 >> v1:172.21.15.169:6789/0 conn(0x7f63dc072420 legacy=0x7f63dc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.101+0000 7f63da800640 1 -- 172.21.15.32:0/982803612 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc0cd5b0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.102+0000 7f63da800640 1 --2- 172.21.15.32:0/982803612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009e50 tx=0x7f63d402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfec0a2943786e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.103+0000 7f63d8a00640 1 -- 172.21.15.32:0/982803612 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d402cb00 con 0x7f63dc0cebb0 2024-04-12T19:51:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.103+0000 7f63d8a00640 1 -- 172.21.15.32:0/982803612 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d402cca0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.103+0000 7f63d8a00640 1 -- 172.21.15.32:0/982803612 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4034610 con 0x7f63dc0cebb0 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 -- 172.21.15.32:0/982803612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 msgr2=0x7f63dc06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 --2- 172.21.15.32:0/982803612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009e50 tx=0x7f63d402bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 -- 172.21.15.32:0/982803612 shutdown_connections 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 --2- 172.21.15.32:0/982803612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc0cebb0 0x7f63dc06df20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 --2- 172.21.15.32:0/982803612 >> v2:172.21.15.169:3300/0 conn(0x7f63dc1461c0 0x7f63dc071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 -- 172.21.15.32:0/982803612 >> 172.21.15.32:0/982803612 conn(0x7f63dc06b930 msgr2=0x7f63dc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 -- 172.21.15.32:0/982803612 shutdown_connections 2024-04-12T19:51:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.104+0000 7f63db200640 1 -- 172.21.15.32:0/982803612 wait complete. 2024-04-12T19:51:25.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.106+0000 7f63db200640 1 Processor -- start 2024-04-12T19:51:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.106+0000 7f63db200640 1 -- start start 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc072420 0x7f63dc1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63dc1461c0 0x7f63dc1f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc0cdb70 con 0x7f63dc0cebb0 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc0cd080 con 0x7f63dc1461c0 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc06e460 con 0x7f63dc072420 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63d9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63d9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44518/0 (socket says 172.21.15.32:44518) 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63d9400640 1 -- 172.21.15.32:0/689831698 learned_addr learned my addr 172.21.15.32:0/689831698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63d9e00640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc072420 0x7f63dc1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.107+0000 7f63da800640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63dc1461c0 0x7f63dc1f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 -- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc072420 msgr2=0x7f63dc1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc072420 0x7f63dc1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 -- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63dc1461c0 msgr2=0x7f63dc1f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63dc1461c0 0x7f63dc1f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 -- 172.21.15.32:0/689831698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc15b9f0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d9400640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c400 tx=0x7f63c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c80052d0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c8005490 con 0x7f63dc0cebb0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007260 con 0x7f63dc0cebb0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc0aa270 con 0x7f63dc0cebb0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.108+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc15bcb0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.110+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc0755c0 con 0x7f63dc0cebb0 2024-04-12T19:51:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.111+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63c8007400 con 0x7f63dc0cebb0 2024-04-12T19:51:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.112+0000 7f63d2000640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 0x7f63bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.112+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7f63c808b150 con 0x7f63dc0cebb0 2024-04-12T19:51:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.112+0000 7f63d9e00640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 0x7f63bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.113+0000 7f63d9e00640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 0x7f63bc06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009a30 tx=0x7f63cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.117+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63c8059490 con 0x7f63dc0cebb0 2024-04-12T19:51:25.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.358+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f63dc15c650 con 0x7f63dc0cebb0 2024-04-12T19:51:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.849+0000 7f63d2000640 1 -- 172.21.15.32:0/689831698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f63c8058d90 con 0x7f63dc0cebb0 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 msgr2=0x7f63bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 0x7f63bc06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009a30 tx=0x7f63cc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 msgr2=0x7f63dc1f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c400 tx=0x7f63c800c8c0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 shutdown_connections 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f63bc06c920 0x7f63bc06ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63dc1461c0 0x7f63dc1f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63dc0cebb0 0x7f63dc1f5cb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 --2- 172.21.15.32:0/689831698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63dc072420 0x7f63dc1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.852+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 >> 172.21.15.32:0/689831698 conn(0x7f63dc06b930 msgr2=0x7f63dc06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.853+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 shutdown_connections 2024-04-12T19:51:25.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:25.853+0000 7f63db200640 1 -- 172.21.15.32:0/689831698 wait complete. 2024-04-12T19:51:25.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-12T19:51:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.111+0000 7f5f1b000640 1 Processor -- start 2024-04-12T19:51:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.111+0000 7f5f1b000640 1 -- start start 2024-04-12T19:51:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f14071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f14072450 0x7f5f1415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f1415d250 0x7f5f1406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1413f9e0 con 0x7f5f1415d250 2024-04-12T19:51:26.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5f14072890 con 0x7f5f14072450 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1413fc70 con 0x7f5f140cec60 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.113+0000 7f5f1a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f1415d250 0x7f5f1406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f1415d250 0x7f5f1406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44530/0 (socket says 172.21.15.32:44530) 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 -- 172.21.15.32:0/3291605272 learned_addr learned my addr 172.21.15.32:0/3291605272 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f19c00640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f14071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f19200640 1 --2- 172.21.15.32:0/3291605272 >> v2:172.21.15.169:3300/0 conn(0x7f5f14072450 0x7f5f1415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 -- 172.21.15.32:0/3291605272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 msgr2=0x7f5f14071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f14071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 -- 172.21.15.32:0/3291605272 >> v2:172.21.15.169:3300/0 conn(0x7f5f14072450 msgr2=0x7f5f1415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 --2- 172.21.15.32:0/3291605272 >> v2:172.21.15.169:3300/0 conn(0x7f5f14072450 0x7f5f1415cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.114+0000 7f5f1a600640 1 -- 172.21.15.32:0/3291605272 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1415da40 con 0x7f5f1415d250 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.115+0000 7f5f1a600640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f1415d250 0x7f5f1406f110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5f140b5a70 tx=0x7f5f0c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d9743b75597a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.115+0000 7f5f13e00640 1 -- 172.21.15.32:0/3291605272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c02cb00 con 0x7f5f1415d250 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.115+0000 7f5f13e00640 1 -- 172.21.15.32:0/3291605272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0c02cca0 con 0x7f5f1415d250 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.115+0000 7f5f13e00640 1 -- 172.21.15.32:0/3291605272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c034610 con 0x7f5f1415d250 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 -- 172.21.15.32:0/3291605272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f1415d250 msgr2=0x7f5f1406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f1415d250 0x7f5f1406f110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5f140b5a70 tx=0x7f5f0c02bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 -- 172.21.15.32:0/3291605272 shutdown_connections 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f1415d250 0x7f5f1406f110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3291605272 >> v2:172.21.15.169:3300/0 conn(0x7f5f14072450 0x7f5f1415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3291605272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f14071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 -- 172.21.15.32:0/3291605272 >> 172.21.15.32:0/3291605272 conn(0x7f5f140b6170 msgr2=0x7f5f1406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 -- 172.21.15.32:0/3291605272 shutdown_connections 2024-04-12T19:51:26.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.116+0000 7f5f1b000640 1 -- 172.21.15.32:0/3291605272 wait complete. 2024-04-12T19:51:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.118+0000 7f5f1b000640 1 Processor -- start 2024-04-12T19:51:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.118+0000 7f5f1b000640 1 -- start start 2024-04-12T19:51:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f140b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1415d250 0x7f5f140b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141404d0 con 0x7f5f14072450 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141401f0 con 0x7f5f1415d250 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f1b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1406de40 con 0x7f5f140cec60 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f19c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f19c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44544/0 (socket says 172.21.15.32:44544) 2024-04-12T19:51:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.119+0000 7f5f19c00640 1 -- 172.21.15.32:0/3815587617 learned_addr learned my addr 172.21.15.32:0/3815587617 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19200640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f140b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f1a600640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1415d250 0x7f5f140b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 -- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 msgr2=0x7f5f140b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f140b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 -- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1415d250 msgr2=0x7f5f140b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1415d250 0x7f5f140b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 -- 172.21.15.32:0/3815587617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f140b1110 con 0x7f5f14072450 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f19c00640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d830 tx=0x7f5f0800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800b6c0 con 0x7f5f14072450 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f08019070 con 0x7f5f14072450 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.120+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800ba80 con 0x7f5f14072450 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.121+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f140b13b0 con 0x7f5f14072450 2024-04-12T19:51:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.121+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f140b1970 con 0x7f5f14072450 2024-04-12T19:51:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.122+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14068b40 con 0x7f5f14072450 2024-04-12T19:51:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.123+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f0801e460 con 0x7f5f14072450 2024-04-12T19:51:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.124+0000 7f5f11600640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.124+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7f5f0808c220 con 0x7f5f14072450 2024-04-12T19:51:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.124+0000 7f5f19200640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.125+0000 7f5f19200640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f140b26c0 tx=0x7f5f00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.128+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f0805a5e0 con 0x7f5f14072450 2024-04-12T19:51:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:26.333+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f140730e0 con 0x7f5f14072450 2024-04-12T19:51:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.053+0000 7f5f11600640 1 -- 172.21.15.32:0/3815587617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f5f08059ee0 con 0x7f5f14072450 2024-04-12T19:51:27.054 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 msgr2=0x7f5ef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f140b26c0 tx=0x7f5f00008040 comp rx=0 tx=0).stop 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 msgr2=0x7f5f140b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d830 tx=0x7f5f0800dcf0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 shutdown_connections 2024-04-12T19:51:27.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ef406c8f0 0x7f5ef406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1415d250 0x7f5f140b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f140cec60 0x7f5f140b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 --2- 172.21.15.32:0/3815587617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f14072450 0x7f5f140b0bd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 >> 172.21.15.32:0/3815587617 conn(0x7f5f140b6170 msgr2=0x7f5f1406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.057+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 shutdown_connections 2024-04-12T19:51:27.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.058+0000 7f5f1b000640 1 -- 172.21.15.32:0/3815587617 wait complete. 2024-04-12T19:51:27.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-12T19:51:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.306+0000 7fa2f1600640 1 Processor -- start 2024-04-12T19:51:27.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.306+0000 7fa2f1600640 1 -- start start 2024-04-12T19:51:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.307+0000 7fa2f1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2ec0cec60 0x7fa2ec071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.307+0000 7fa2f1600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2ec13fc70 con 0x7fa2ec15d420 2024-04-12T19:51:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.307+0000 7fa2f1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2ec141a00 con 0x7fa2ec072450 2024-04-12T19:51:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.307+0000 7fa2f1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2ec0728c0 con 0x7fa2ec0cec60 2024-04-12T19:51:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.308+0000 7fa2f0c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2ec15d420 0x7fa2ec06f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56642/0 (socket says 172.21.15.32:56642) 2024-04-12T19:51:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.308+0000 7fa2f0c00640 1 -- 172.21.15.32:0/1479678827 learned_addr learned my addr 172.21.15.32:0/1479678827 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.308+0000 7fa2ebe00640 1 --2- 172.21.15.32:0/1479678827 >> v2:172.21.15.169:3300/0 conn(0x7fa2ec0cec60 0x7fa2ec071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.308+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106849777 0 0) 0x7fa2ec13fc70 con 0x7fa2ec15d420 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.308+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2d0003520 con 0x7fa2ec15d420 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2008305390 0 0) 0x7fa2d0003520 con 0x7fa2ec15d420 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 >> v2:172.21.15.169:3300/0 conn(0x7fa2ec0cec60 msgr2=0x7fa2ec071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 --2- 172.21.15.32:0/1479678827 >> v2:172.21.15.169:3300/0 conn(0x7fa2ec0cec60 0x7fa2ec071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 >> v1:172.21.15.169:6789/0 conn(0x7fa2ec072450 legacy=0x7fa2ec15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec15da40 con 0x7fa2ec15d420 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fa2e40019f0 con 0x7fa2ec15d420 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 >> v1:172.21.15.32:6789/0 conn(0x7fa2ec15d420 legacy=0x7fa2ec06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00039b0 0x7fa2d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2d0004800 0x7fa2d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 0x7fa2d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec0728c0 con 0x7fa2d00051c0 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000c070 con 0x7fa2d0004800 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d000c270 con 0x7fa2d00039b0 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2f0c00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 0x7fa2d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.309+0000 7fa2e3e00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2d0004800 0x7fa2d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2ebe00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00039b0 0x7fa2d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2f0c00640 1 -- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00039b0 msgr2=0x7fa2d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2f0c00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00039b0 0x7fa2d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2f0c00640 1 -- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2d0004800 msgr2=0x7fa2d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2f0c00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2d0004800 0x7fa2d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2f0c00640 1 -- 172.21.15.32:0/1479678827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d0003520 con 0x7fa2d00051c0 2024-04-12T19:51:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.310+0000 7fa2ebe00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00039b0 0x7fa2d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.311+0000 7fa2f0c00640 1 -- 172.21.15.32:0/1479678827 reap_dead start 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.311+0000 7fa2f0c00640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 0x7fa2d000b9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400a850 tx=0x7fa2d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6696457e8f6d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.311+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400a4a0 con 0x7fa2d00051c0 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.311+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e4034650 con 0x7fa2d00051c0 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.311+0000 7fa2eb400640 1 -- 172.21.15.32:0/1479678827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4034a40 con 0x7fa2d00051c0 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 -- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 msgr2=0x7fa2d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 0x7fa2d000b9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400a850 tx=0x7fa2d0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 -- 172.21.15.32:0/1479678827 shutdown_connections 2024-04-12T19:51:27.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 --2- 172.21.15.32:0/1479678827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2d00051c0 0x7fa2d000b9c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 -- 172.21.15.32:0/1479678827 >> 172.21.15.32:0/1479678827 conn(0x7fa2ec0b6170 msgr2=0x7fa2ec06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 -- 172.21.15.32:0/1479678827 shutdown_connections 2024-04-12T19:51:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.312+0000 7fa2f1600640 1 -- 172.21.15.32:0/1479678827 wait complete. 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.314+0000 7fa2f1600640 1 Processor -- start 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.314+0000 7fa2f1600640 1 -- start start 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00051c0 0x7fa2ec1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ec15ac50 0x7fa2ec15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1401f0 con 0x7fa2ec0af140 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec1404d0 con 0x7fa2ec15ac50 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ec06de40 con 0x7fa2d00051c0 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44556/0 (socket says 172.21.15.32:44556) 2024-04-12T19:51:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 -- 172.21.15.32:0/2811685714 learned_addr learned my addr 172.21.15.32:0/2811685714 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2f0c00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ec15ac50 0x7fa2ec15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 -- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00051c0 msgr2=0x7fa2ec1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00051c0 0x7fa2ec1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 -- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ec15ac50 msgr2=0x7fa2ec15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ec15ac50 0x7fa2ec15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.315+0000 7fa2e3e00640 1 -- 172.21.15.32:0/2811685714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec158770 con 0x7fa2ec0af140 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2e3e00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e480 tx=0x7fa2d400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40052e0 con 0x7fa2ec0af140 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4018070 con 0x7fa2ec0af140 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4003830 con 0x7fa2ec0af140 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec159d00 con 0x7fa2ec0af140 2024-04-12T19:51:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.316+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec157ed0 con 0x7fa2ec0af140 2024-04-12T19:51:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.317+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec068b40 con 0x7fa2ec0af140 2024-04-12T19:51:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.318+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2d40039d0 con 0x7fa2ec0af140 2024-04-12T19:51:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.319+0000 7fa2e8c00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.319+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7fa2d408b6e0 con 0x7fa2ec0af140 2024-04-12T19:51:27.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.320+0000 7fa2ebe00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.320+0000 7fa2ebe00640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0099e0 tx=0x7fa2dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.324+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2d4059c40 con 0x7fa2ec0af140 2024-04-12T19:51:27.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:27.530+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa2ec0730e0 con 0x7fa2ec0af140 2024-04-12T19:51:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.063+0000 7fa2e8c00640 1 -- 172.21.15.32:0/2811685714 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7fa2d4059540 con 0x7fa2ec0af140 2024-04-12T19:51:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-12T19:51:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 msgr2=0x7fa2c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0099e0 tx=0x7fa2dc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 msgr2=0x7fa2ec157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e480 tx=0x7fa2d400e940 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 shutdown_connections 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2ec15ac50 0x7fa2ec15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ec0af140 0x7fa2ec157280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 --2- 172.21.15.32:0/2811685714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2d00051c0 0x7fa2ec1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.069+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 >> 172.21.15.32:0/2811685714 conn(0x7fa2ec0b6170 msgr2=0x7fa2ec06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.070+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 shutdown_connections 2024-04-12T19:51:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.070+0000 7fa2f1600640 1 -- 172.21.15.32:0/2811685714 wait complete. 2024-04-12T19:51:28.098 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-12T19:51:28.098 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-12T19:51:28.098 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-12T19:51:28.098 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:51:28.098 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-12T19:51:28.099 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-12T19:51:28.099 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-12T19:51:28.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:51:28.099 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-12T19:51:28.099 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-12T19:51:28.099 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-12T19:51:28.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:51:28.100 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-12T19:51:28.100 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-12T19:51:28.100 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-12T19:51:28.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:51:28.119 INFO:tasks.ceph.mds.c:Stopped 2024-04-12T19:51:28.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-12T19:51:28.122 INFO:tasks.ceph.mds.b:Stopped 2024-04-12T19:51:28.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-12T19:51:28.123 INFO:tasks.ceph.mds.a:Stopped 2024-04-12T19:51:28.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-12T19:51:28.123 INFO:tasks.ceph.mds.d:Stopped 2024-04-12T19:51:28.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-12T19:51:28.168 INFO:tasks.ceph.mds.c:Started 2024-04-12T19:51:28.169 DEBUG:teuthology.parallel:result is None 2024-04-12T19:51:28.170 INFO:tasks.ceph.mds.b:Started 2024-04-12T19:51:28.170 DEBUG:teuthology.parallel:result is None 2024-04-12T19:51:28.175 INFO:tasks.ceph.mds.a:Started 2024-04-12T19:51:28.175 DEBUG:teuthology.parallel:result is None 2024-04-12T19:51:28.175 INFO:tasks.ceph.mds.d:Started 2024-04-12T19:51:28.176 DEBUG:teuthology.parallel:result is None 2024-04-12T19:51:28.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-12T19:51:28.332 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-12T19:51:28.333 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-12T19:51:28.333 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-12T19:51:28.333 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-12T19:51:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.374+0000 7fec76c00640 1 Processor -- start 2024-04-12T19:51:28.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.375+0000 7fec76c00640 1 -- start start 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec76c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec70148110 0x7fec7015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec76c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7015ff20 con 0x7fec7015f4e0 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec76c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7015f960 con 0x7fec700b43b0 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec76c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fec7015fc40 con 0x7fec70148110 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec6f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec7015f4e0 0x7fec700aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56658/0 (socket says 172.21.15.32:56658) 2024-04-12T19:51:28.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec6f400640 1 -- 172.21.15.32:0/616135077 learned_addr learned my addr 172.21.15.32:0/616135077 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:28.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.376+0000 7fec6fe00640 1 --2- 172.21.15.32:0/616135077 >> v2:172.21.15.169:3300/0 conn(0x7fec70148110 0x7fec7015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3117531667 0 0) 0x7fec7015ff20 con 0x7fec7015f4e0 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec54003520 con 0x7fec7015f4e0 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047402318 0 0) 0x7fec7015f960 con 0x7fec700b43b0 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec7015ff20 con 0x7fec700b43b0 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6fe00640 1 -- 172.21.15.32:0/616135077 >> v1:172.21.15.169:6789/0 conn(0x7fec700b43b0 legacy=0x7fec70072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6fe00640 1 -- 172.21.15.32:0/616135077 >> v1:172.21.15.32:6789/0 conn(0x7fec7015f4e0 legacy=0x7fec700aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.377+0000 7fec6fe00640 1 -- 172.21.15.32:0/616135077 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec700b4b20 con 0x7fec70148110 2024-04-12T19:51:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.379+0000 7fec6fe00640 1 --2- 172.21.15.32:0/616135077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec70148110 0x7fec7015efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fec600097d0 tx=0x7fec6002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e7c9fdf95fb66c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.379+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60034070 con 0x7fec70148110 2024-04-12T19:51:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.379+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60030800 con 0x7fec70148110 2024-04-12T19:51:28.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.379+0000 7fec6ea00640 1 -- 172.21.15.32:0/616135077 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60030bd0 con 0x7fec70148110 2024-04-12T19:51:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.379+0000 7fec76c00640 1 -- 172.21.15.32:0/616135077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec70148110 msgr2=0x7fec7015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 --2- 172.21.15.32:0/616135077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec70148110 0x7fec7015efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fec600097d0 tx=0x7fec6002b940 comp rx=0 tx=0).stop 2024-04-12T19:51:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 -- 172.21.15.32:0/616135077 shutdown_connections 2024-04-12T19:51:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 --2- 172.21.15.32:0/616135077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec70148110 0x7fec7015efa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 -- 172.21.15.32:0/616135077 >> 172.21.15.32:0/616135077 conn(0x7fec700b2460 msgr2=0x7fec700b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 -- 172.21.15.32:0/616135077 shutdown_connections 2024-04-12T19:51:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.380+0000 7fec76c00640 1 -- 172.21.15.32:0/616135077 wait complete. 2024-04-12T19:51:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.382+0000 7fec76c00640 1 Processor -- start 2024-04-12T19:51:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.382+0000 7fec76c00640 1 -- start start 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700b43b0 0x7fec7015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec70148110 0x7fec70130060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec700af3c0 con 0x7fec7015f4e0 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec70148570 con 0x7fec700b43b0 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec76c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec700af540 con 0x7fec70148110 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec74c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec74c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44600/0 (socket says 172.21.15.32:44600) 2024-04-12T19:51:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec74c00640 1 -- 172.21.15.32:0/4123276150 learned_addr learned my addr 172.21.15.32:0/4123276150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec6fe00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700b43b0 0x7fec7015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.383+0000 7fec6f400640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec70148110 0x7fec70130060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 -- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec70148110 msgr2=0x7fec70130060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec70148110 0x7fec70130060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 -- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700b43b0 msgr2=0x7fec7015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700b43b0 0x7fec7015e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 -- 172.21.15.32:0/4123276150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec70130690 con 0x7fec7015f4e0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec74c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c7d0 tx=0x7fec6800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68003b70 con 0x7fec7015f4e0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec68003d10 con 0x7fec7015f4e0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800aa90 con 0x7fec7015f4e0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.384+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec70130970 con 0x7fec7015f4e0 2024-04-12T19:51:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.385+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec70130f30 con 0x7fec7015f4e0 2024-04-12T19:51:28.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.386+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec70068c80 con 0x7fec7015f4e0 2024-04-12T19:51:28.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.387+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec6800ac30 con 0x7fec7015f4e0 2024-04-12T19:51:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.387+0000 7fec67e00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.388+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7fec6808b670 con 0x7fec7015f4e0 2024-04-12T19:51:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.388+0000 7fec6fe00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.389+0000 7fec6fe00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 0x7fec5006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec60036000 tx=0x7fec6003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.392+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec68059740 con 0x7fec7015f4e0 2024-04-12T19:51:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:28.607+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fec701e59c0 con 0x7fec7015f4e0 2024-04-12T19:51:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.087+0000 7fec67e00640 1 -- 172.21.15.32:0/4123276150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7fec6805da40 con 0x7fec7015f4e0 2024-04-12T19:51:29.088 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-12T19:51:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 msgr2=0x7fec5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 0x7fec5006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec60036000 tx=0x7fec6003b040 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 msgr2=0x7fec701e5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c7d0 tx=0x7fec6800cc90 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 shutdown_connections 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fec5006c8f0 0x7fec5006edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec7015f4e0 0x7fec701e5480 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec70148110 0x7fec70130060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 --2- 172.21.15.32:0/4123276150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700b43b0 0x7fec7015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 >> 172.21.15.32:0/4123276150 conn(0x7fec700b2460 msgr2=0x7fec700af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.092+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 shutdown_connections 2024-04-12T19:51:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.093+0000 7fec76c00640 1 -- 172.21.15.32:0/4123276150 wait complete. 2024-04-12T19:51:29.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-12T19:51:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.328+0000 7f155f200640 1 Processor -- start 2024-04-12T19:51:29.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.328+0000 7f155f200640 1 -- start start 2024-04-12T19:51:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.329+0000 7f155f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15600cf110 0x7f1560162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.329+0000 7f155f200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15600d0090 con 0x7f1560162020 2024-04-12T19:51:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.329+0000 7f155f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15600cfad0 con 0x7f15600d1c90 2024-04-12T19:51:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.329+0000 7f155f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1560163440 con 0x7f15600cf110 2024-04-12T19:51:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155de00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1560162020 0x7f15600d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56660/0 (socket says 172.21.15.32:56660) 2024-04-12T19:51:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155de00640 1 -- 172.21.15.32:0/652364231 learned_addr learned my addr 172.21.15.32:0/652364231 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155e800640 1 --2- 172.21.15.32:0/652364231 >> v2:172.21.15.169:3300/0 conn(0x7f15600cf110 0x7f1560162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044048399 0 0) 0x7f1550001110 con 0x7f1560162020 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1540003520 con 0x7f1560162020 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 599854602 0 0) 0x7f1540003520 con 0x7f1560162020 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 >> v2:172.21.15.169:3300/0 conn(0x7f15600cf110 msgr2=0x7f1560162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 --2- 172.21.15.32:0/652364231 >> v2:172.21.15.169:3300/0 conn(0x7f15600cf110 0x7f1560162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.330+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 >> v1:172.21.15.169:6789/0 conn(0x7f15600d1c90 legacy=0x7f15600d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600cfdb0 con 0x7f1560162020 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f1550001840 con 0x7f1560162020 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 >> v1:172.21.15.32:6789/0 conn(0x7f1560162020 legacy=0x7f15600d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1540003a00 0x7f1540003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f1540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1540005210 0x7f154000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1560163440 con 0x7f1540004800 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f154000c160 con 0x7f1540005210 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f154000c360 con 0x7f1540003a00 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155d400640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f1540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155e800640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1540005210 0x7f154000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.331+0000 7f155de00640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1540003a00 0x7f1540003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155d400640 1 -- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1540003a00 msgr2=0x7f1540003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155d400640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1540003a00 0x7f1540003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155d400640 1 -- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1540005210 msgr2=0x7f154000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155d400640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1540005210 0x7f154000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155d400640 1 -- 172.21.15.32:0/652364231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1540003520 con 0x7f1540004800 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155e800640 1 -- 172.21.15.32:0/652364231 reap_dead start 2024-04-12T19:51:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.332+0000 7f155de00640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1540003a00 0x7f1540003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:29.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.333+0000 7f155d400640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f1540004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1548001e50 tx=0x7f1540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1d0c148994a9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.333+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548001a70 con 0x7f1540004800 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.333+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1548003b70 con 0x7f1540004800 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.333+0000 7f155ca00640 1 -- 172.21.15.32:0/652364231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480050d0 con 0x7f1540004800 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 -- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 msgr2=0x7f1540004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f1540004c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1548001e50 tx=0x7f1540005830 comp rx=0 tx=0).stop 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 -- 172.21.15.32:0/652364231 shutdown_connections 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 --2- 172.21.15.32:0/652364231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f1540004c80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 -- 172.21.15.32:0/652364231 >> 172.21.15.32:0/652364231 conn(0x7f156006cf00 msgr2=0x7f156006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 -- 172.21.15.32:0/652364231 shutdown_connections 2024-04-12T19:51:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.334+0000 7f155f200640 1 -- 172.21.15.32:0/652364231 wait complete. 2024-04-12T19:51:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.336+0000 7f155f200640 1 Processor -- start 2024-04-12T19:51:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.336+0000 7f155f200640 1 -- start start 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1560157d80 0x7f15601f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601f5ea0 0x7f15601f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f15600cf880 con 0x7f1540004800 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15600b7b30 con 0x7f15601f5ea0 2024-04-12T19:51:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15600d2e70 con 0x7f1560157d80 2024-04-12T19:51:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44628/0 (socket says 172.21.15.32:44628) 2024-04-12T19:51:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.337+0000 7f155de00640 1 -- 172.21.15.32:0/1296257570 learned_addr learned my addr 172.21.15.32:0/1296257570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 -- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1560157d80 msgr2=0x7f15601f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1560157d80 0x7f15601f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 -- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601f5ea0 msgr2=0x7f15601f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155e800640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601f5ea0 0x7f15601f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601f5ea0 0x7f15601f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 -- 172.21.15.32:0/1296257570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600cc940 con 0x7f1540004800 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155de00640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f155000d7a0 tx=0x7f155000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550019070 con 0x7f1540004800 2024-04-12T19:51:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15500074e0 con 0x7f1540004800 2024-04-12T19:51:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550004a00 con 0x7f1540004800 2024-04-12T19:51:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.338+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15600ccbe0 con 0x7f1540004800 2024-04-12T19:51:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.339+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15600cd1a0 con 0x7f1540004800 2024-04-12T19:51:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.340+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15600755c0 con 0x7f1540004800 2024-04-12T19:51:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.341+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f155001c070 con 0x7f1540004800 2024-04-12T19:51:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.342+0000 7f1556000640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 0x7f153c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.342+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f155008b270 con 0x7f1540004800 2024-04-12T19:51:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.342+0000 7f155d400640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 0x7f153c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.343+0000 7f155d400640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 0x7f153c06ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1548034000 tx=0x7f1548030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.346+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15600d0090 con 0x7f1540004800 2024-04-12T19:51:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.623+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15600b7650 con 0x7f1540004800 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.624+0000 7f1556000640 1 -- 172.21.15.32:0/1296257570 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f15500599e0 con 0x7f1540004800 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-12T19:51:29.624 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA5kBlm+J8zLhAASItkJ07VRlCtU+nHBrM6WQ==", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA6kBlm+AuDFRAAi1aBIRh5rbqDOpn60uPFlg==", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-12T19:51:29.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA6kBlmEwWhBBAAicXm9WD14N+M24IV0CNtGg==", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA6kBlmlAtBJRAAoou+uSYwQdHVlmLEJu8i0Q==", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA9kBlmHjIdFxAARX/JJCf/osbmHZRsPfA9WQ==", 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBAkBlmwcJgNBAAT/iZHIYj+laliEVa/pJ5Wg==", 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBEkBlm9HtZCBAAjbA85CAONlb+W4zAWJhtxA==", 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.629 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBHkBlmr1GyJRAAT6SHhRDHg5kRUwht7VUGmw==", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBOkBlmDJqyBBAAZupf7xG9Jo9iPDelxGgbtw==", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRkBlmuDCwIBAAAqfFnzUE8ONPfSv/BB6JOg==", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBUkBlm70egLhAAYfk0Z4Im7bZjQ4nDSdEKXA==", 2024-04-12T19:51:29.631 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBYkBlmCDnbDRAA46U5s4hIaBw4rk+7w4We7A==", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA6kBlmq9xnMxAA2e+ydW8KXvsNYpAvO4M4Xg==", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7kBlmouBwDhAAVSaj/2Rp875osbza2jrMhw==", 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.633 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7kBlmFu6iAhAAdZny4smaQk5UVNj+ZmZQsg==", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-12T19:51:29.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7kBlmYCpcGRAAfp52MuYpAUedSJCAAWVp1A==", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4kBlmMatPHxAAAinzd6pQwLu9OfCw/vafIw==", 2024-04-12T19:51:29.635 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmNiT9EBAAjn1xZcoMcbGjmLtXiEN3uw==", 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.636 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmKEH9EBAAi/AUczAQPWbR+GoEEn+bkg==", 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmclr9EBAA2lO6YhqgmEXGWPCp18CZbg==", 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2024-04-12T19:51:29.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmtHP9EBAAdnGhbGGR+PDL7YlhUsFQFw==", 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmv5P9EBAAooEBcHFfHbiVqUngMomtWQ==", 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-12T19:51:29.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBmkBlmNK79EBAAQfA/V9qA7d5jzqRCF9YNbg==", 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA5kBlmyoWLIRAAFt1Wj5jPNanY8rFNEH0yRg==", 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-12T19:51:29.639 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA5kBlmrLL3FBAAe6dbs5ttGba1qoSrVBLpMw==", 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-12T19:51:29.640 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:51:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.626+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 msgr2=0x7f153c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.626+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 0x7f153c06ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1548034000 tx=0x7f1548030000 comp rx=0 tx=0).stop 2024-04-12T19:51:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.626+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 msgr2=0x7f15600cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.626+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f155000d7a0 tx=0x7f155000dc60 comp rx=0 tx=0).stop 2024-04-12T19:51:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 shutdown_connections 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f153c06c920 0x7f153c06ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601f5ea0 0x7f15601f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1560157d80 0x7f15601f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 --2- 172.21.15.32:0/1296257570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540004800 0x7f15600cdc80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 >> 172.21.15.32:0/1296257570 conn(0x7f156006cf00 msgr2=0x7f156015ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 shutdown_connections 2024-04-12T19:51:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.627+0000 7f155f200640 1 -- 172.21.15.32:0/1296257570 wait complete. 2024-04-12T19:51:29.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-12T19:51:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.861+0000 7fa687200640 1 Processor -- start 2024-04-12T19:51:29.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.861+0000 7fa687200640 1 -- start start 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.862+0000 7fa687200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68815eb40 0x7fa688162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa687200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6880d1650 0x7fa6880d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa687200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa688163400 con 0x7fa6880ba460 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa687200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa688162e40 con 0x7fa6880d1650 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa687200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa688163120 con 0x7fa68815eb40 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa686800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6880d1650 0x7fa6880d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa686800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6880d1650 0x7fa6880d33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44644/0 (socket says 172.21.15.32:44644) 2024-04-12T19:51:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa686800640 1 -- 172.21.15.32:0/72846029 learned_addr learned my addr 172.21.15.32:0/72846029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.863+0000 7fa685e00640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68815eb40 0x7fa688162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.864+0000 7fa686800640 1 -- 172.21.15.32:0/72846029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68815eb40 msgr2=0x7fa688162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.864+0000 7fa686800640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68815eb40 0x7fa688162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.864+0000 7fa686800640 1 -- 172.21.15.32:0/72846029 >> v1:172.21.15.169:6789/0 conn(0x7fa6880ba460 legacy=0x7fa6880d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:29.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.864+0000 7fa686800640 1 -- 172.21.15.32:0/72846029 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6880d2bc0 con 0x7fa6880d1650 2024-04-12T19:51:29.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.865+0000 7fa686800640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880d1650 0x7fa6880d33c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa680009870 tx=0x7fa68002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868294d933ec673e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.865+0000 7fa684a00640 1 -- 172.21.15.32:0/72846029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800317c0 con 0x7fa6880d1650 2024-04-12T19:51:29.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.865+0000 7fa684a00640 1 -- 172.21.15.32:0/72846029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680037070 con 0x7fa6880d1650 2024-04-12T19:51:29.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.865+0000 7fa684a00640 1 -- 172.21.15.32:0/72846029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680031b80 con 0x7fa6880d1650 2024-04-12T19:51:29.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 -- 172.21.15.32:0/72846029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880d1650 msgr2=0x7fa6880d33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880d1650 0x7fa6880d33c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa680009870 tx=0x7fa68002b860 comp rx=0 tx=0).stop 2024-04-12T19:51:29.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 -- 172.21.15.32:0/72846029 shutdown_connections 2024-04-12T19:51:29.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880d1650 0x7fa6880d33c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 --2- 172.21.15.32:0/72846029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa68815eb40 0x7fa688162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 -- 172.21.15.32:0/72846029 >> 172.21.15.32:0/72846029 conn(0x7fa68806d3d0 msgr2=0x7fa688073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:29.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 -- 172.21.15.32:0/72846029 shutdown_connections 2024-04-12T19:51:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.866+0000 7fa687200640 1 -- 172.21.15.32:0/72846029 wait complete. 2024-04-12T19:51:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.868+0000 7fa687200640 1 Processor -- start 2024-04-12T19:51:29.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.868+0000 7fa687200640 1 -- start start 2024-04-12T19:51:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6880d1650 0x7fa6880cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68815eb40 0x7fa688159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa688162a50 con 0x7fa6880ba460 2024-04-12T19:51:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880b7860 con 0x7fa68815eb40 2024-04-12T19:51:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.869+0000 7fa687200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa688162bd0 con 0x7fa6880d1650 2024-04-12T19:51:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44646/0 (socket says 172.21.15.32:44646) 2024-04-12T19:51:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 -- 172.21.15.32:0/3654937918 learned_addr learned my addr 172.21.15.32:0/3654937918 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:29.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685400640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6880d1650 0x7fa6880cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa686800640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68815eb40 0x7fa688159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 -- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6880d1650 msgr2=0x7fa6880cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6880d1650 0x7fa6880cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 -- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68815eb40 msgr2=0x7fa688159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68815eb40 0x7fa688159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 -- 172.21.15.32:0/3654937918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6880ce210 con 0x7fa6880ba460 2024-04-12T19:51:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.870+0000 7fa685e00640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa67800d620 tx=0x7fa67800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.871+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800b620 con 0x7fa6880ba460 2024-04-12T19:51:29.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.871+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa678019070 con 0x7fa6880ba460 2024-04-12T19:51:29.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.871+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800ba00 con 0x7fa6880ba460 2024-04-12T19:51:29.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.871+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6880cb800 con 0x7fa6880ba460 2024-04-12T19:51:29.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.871+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6880cc0a0 con 0x7fa6880ba460 2024-04-12T19:51:29.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.872+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6880cb9d0 con 0x7fa6880ba460 2024-04-12T19:51:29.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.873+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa67800bba0 con 0x7fa6880ba460 2024-04-12T19:51:29.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.874+0000 7fa67e000640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 0x7fa66406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:29.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.874+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fa67808bc90 con 0x7fa6880ba460 2024-04-12T19:51:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.874+0000 7fa685400640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 0x7fa66406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:29.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.875+0000 7fa685400640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 0x7fa66406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6880c8190 tx=0x7fa670009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:29.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:29.878+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa67805a370 con 0x7fa6880ba460 2024-04-12T19:51:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.160+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fa6880755c0 con 0x7fa6880ba460 2024-04-12T19:51:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.164+0000 7fa67e000640 1 -- 172.21.15.32:0/3654937918 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fa678060070 con 0x7fa6880ba460 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.166+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 msgr2=0x7fa66406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.166+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 0x7fa66406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6880c8190 tx=0x7fa670009040 comp rx=0 tx=0).stop 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.166+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 msgr2=0x7fa68815baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.166+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa67800d620 tx=0x7fa67800dae0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 shutdown_connections 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa66406c8f0 0x7fa66406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa68815eb40 0x7fa688159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6880d1650 0x7fa6880cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 --2- 172.21.15.32:0/3654937918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6880ba460 0x7fa68815baf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 >> 172.21.15.32:0/3654937918 conn(0x7fa68806d3d0 msgr2=0x7fa6880cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 shutdown_connections 2024-04-12T19:51:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.167+0000 7fa687200640 1 -- 172.21.15.32:0/3654937918 wait complete. 2024-04-12T19:51:30.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-12T19:51:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.404+0000 7fb343200640 1 Processor -- start 2024-04-12T19:51:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.404+0000 7fb343200640 1 -- start start 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb343200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415b410 0x7fb34415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb343200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3440cf530 0x7fb3440cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb343200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34415cae0 con 0x7fb34415f110 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb343200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440cd360 con 0x7fb3440cf530 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb343200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440cd640 con 0x7fb34415b410 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb341400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3440cf530 0x7fb3440cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb341400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3440cf530 0x7fb3440cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:35876/0 (socket says 172.21.15.32:35876) 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.406+0000 7fb341400640 1 -- 172.21.15.32:0/708389128 learned_addr learned my addr 172.21.15.32:0/708389128 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.407+0000 7fb341e00640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415b410 0x7fb34415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.407+0000 7fb341400640 1 -- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415b410 msgr2=0x7fb34415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.407+0000 7fb341400640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415b410 0x7fb34415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.407+0000 7fb341400640 1 -- 172.21.15.32:0/708389128 >> v1:172.21.15.169:6789/0 conn(0x7fb34415f110 legacy=0x7fb34415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.407+0000 7fb341400640 1 -- 172.21.15.32:0/708389128 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440cd920 con 0x7fb3440cf530 2024-04-12T19:51:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.409+0000 7fb341400640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb3440cf9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76db4c1a13570fad server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.409+0000 7fb340a00640 1 -- 172.21.15.32:0/708389128 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330035020 con 0x7fb3440cf530 2024-04-12T19:51:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.409+0000 7fb340a00640 1 -- 172.21.15.32:0/708389128 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb330031640 con 0x7fb3440cf530 2024-04-12T19:51:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.409+0000 7fb340a00640 1 -- 172.21.15.32:0/708389128 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33002d040 con 0x7fb3440cf530 2024-04-12T19:51:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 -- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 msgr2=0x7fb3440cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb3440cf9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 -- 172.21.15.32:0/708389128 shutdown_connections 2024-04-12T19:51:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb3440cf9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 --2- 172.21.15.32:0/708389128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415b410 0x7fb34415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 -- 172.21.15.32:0/708389128 >> 172.21.15.32:0/708389128 conn(0x7fb3440b23a0 msgr2=0x7fb3440b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 -- 172.21.15.32:0/708389128 shutdown_connections 2024-04-12T19:51:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.410+0000 7fb343200640 1 -- 172.21.15.32:0/708389128 wait complete. 2024-04-12T19:51:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.412+0000 7fb343200640 1 Processor -- start 2024-04-12T19:51:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.412+0000 7fb343200640 1 -- start start 2024-04-12T19:51:30.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb34414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415f110 0x7fb344156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440cff00 con 0x7fb34415b410 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440cdee0 con 0x7fb3440cf530 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb343200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440cd030 con 0x7fb34415f110 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb341400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb341400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44654/0 (socket says 172.21.15.32:44654) 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.413+0000 7fb341400640 1 -- 172.21.15.32:0/1697989528 learned_addr learned my addr 172.21.15.32:0/1697989528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341e00640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb34414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb342800640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415f110 0x7fb344156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 -- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415f110 msgr2=0x7fb344156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415f110 0x7fb344156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 -- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 msgr2=0x7fb34414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb34414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 -- 172.21.15.32:0/1697989528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440b4fb0 con 0x7fb34415b410 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.414+0000 7fb341400640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb33002b930 tx=0x7fb3300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.415+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330040070 con 0x7fb34415b410 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.415+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33003e070 con 0x7fb34415b410 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.415+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34413a130 con 0x7fb34415b410 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.415+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330035040 con 0x7fb34415b410 2024-04-12T19:51:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.415+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34414e150 con 0x7fb34415b410 2024-04-12T19:51:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.416+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3440755c0 con 0x7fb34415b410 2024-04-12T19:51:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.417+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb330038020 con 0x7fb34415b410 2024-04-12T19:51:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.418+0000 7fb33a000640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 0x7fb32406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.418+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fb3300ad160 con 0x7fb34415b410 2024-04-12T19:51:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.418+0000 7fb341e00640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 0x7fb32406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.419+0000 7fb341e00640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 0x7fb32406ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb34406de70 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.422+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb33007b840 con 0x7fb34415b410 2024-04-12T19:51:30.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.689+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb344151a90 con 0x7fb34415b410 2024-04-12T19:51:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.693+0000 7fb33a000640 1 -- 172.21.15.32:0/1697989528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fb33007b140 con 0x7fb34415b410 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 msgr2=0x7fb32406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 0x7fb32406ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb34406de70 tx=0x7fb334008040 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 msgr2=0x7fb344139bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb33002b930 tx=0x7fb3300046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 shutdown_connections 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb32406c920 0x7fb32406ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb34415f110 0x7fb344156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb34415b410 0x7fb344139bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 --2- 172.21.15.32:0/1697989528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3440cf530 0x7fb34414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.695+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 >> 172.21.15.32:0/1697989528 conn(0x7fb3440b23a0 msgr2=0x7fb34406dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.696+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 shutdown_connections 2024-04-12T19:51:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.696+0000 7fb343200640 1 -- 172.21.15.32:0/1697989528 wait complete. 2024-04-12T19:51:30.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-12T19:51:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.934+0000 7f572fe00640 1 Processor -- start 2024-04-12T19:51:30.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.934+0000 7f572fe00640 1 -- start start 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57300afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572fe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57300ce760 con 0x7f57300b5b00 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57300cea40 con 0x7f57300b5530 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f57300ced20 con 0x7f5730162dd0 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57300b5b00 0x7f57300b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56674/0 (socket says 172.21.15.32:56674) 2024-04-12T19:51:30.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572ea00640 1 -- 172.21.15.32:0/70805485 learned_addr learned my addr 172.21.15.32:0/70805485 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f5727e00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57300afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.936+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142191488 0 0) 0x7f57300ce760 con 0x7f57300b5b00 2024-04-12T19:51:30.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5714003520 con 0x7f57300b5b00 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1977815407 0 0) 0x7f5714003520 con 0x7f57300b5b00 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 msgr2=0x7f57300afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57300afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 >> v1:172.21.15.169:6789/0 conn(0x7f57300b5530 legacy=0x7f57300b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57300ce410 con 0x7f57300b5b00 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f5720001670 con 0x7f57300b5b00 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 >> v1:172.21.15.32:6789/0 conn(0x7f57300b5b00 legacy=0x7f57300b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57140039e0 0x7f5714003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f5714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57300ced20 con 0x7f5714004800 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f571400baa0 con 0x7f57140039e0 2024-04-12T19:51:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f571400bca0 con 0x7f5730162dd0 2024-04-12T19:51:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.937+0000 7f5727e00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f5714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f572ea00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57140039e0 0x7f5714003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f572f400640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f5727e00640 1 -- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 msgr2=0x7f57140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f5727e00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f5727e00640 1 -- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57140039e0 msgr2=0x7f5714003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f5727e00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57140039e0 0x7f5714003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f5727e00640 1 -- 172.21.15.32:0/70805485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5714003520 con 0x7f5714004800 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.938+0000 7f572f400640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730162dd0 0x7f57140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.939+0000 7f5727e00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f5714004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f57301631d0 tx=0x7f5714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa3cd56776e0734 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.939+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718039070 con 0x7f5714004800 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.939+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571802c630 con 0x7f5714004800 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.939+0000 7f572e000640 1 -- 172.21.15.32:0/70805485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718009690 con 0x7f5714004800 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 -- 172.21.15.32:0/70805485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 msgr2=0x7f5714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 --2- 172.21.15.32:0/70805485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f5714004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f57301631d0 tx=0x7f5714005830 comp rx=0 tx=0).stop 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572f400640 1 -- 172.21.15.32:0/70805485 reap_dead start 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 -- 172.21.15.32:0/70805485 shutdown_connections 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 -- 172.21.15.32:0/70805485 >> 172.21.15.32:0/70805485 conn(0x7f57300b23a0 msgr2=0x7f57300b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 -- 172.21.15.32:0/70805485 shutdown_connections 2024-04-12T19:51:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.940+0000 7f572fe00640 1 -- 172.21.15.32:0/70805485 wait complete. 2024-04-12T19:51:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.942+0000 7f572fe00640 1 Processor -- start 2024-04-12T19:51:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.942+0000 7f572fe00640 1 -- start start 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730149580 0x7f5730146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730157090 0x7f5730157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57300b0180 con 0x7f5714004800 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57300b0000 con 0x7f5730157090 2024-04-12T19:51:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f57300cf520 con 0x7f5730149580 2024-04-12T19:51:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44666/0 (socket says 172.21.15.32:44666) 2024-04-12T19:51:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.943+0000 7f572ea00640 1 -- 172.21.15.32:0/1762355951 learned_addr learned my addr 172.21.15.32:0/1762355951 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 -- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730149580 msgr2=0x7f5730146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730149580 0x7f5730146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 -- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730157090 msgr2=0x7f5730157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730157090 0x7f5730157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 -- 172.21.15.32:0/1762355951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57301467b0 con 0x7f5714004800 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572ea00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f60 tx=0x7f572000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572000c8d0 con 0x7f5714004800 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5720018070 con 0x7f5714004800 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572000ccd0 con 0x7f5714004800 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.944+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573015dab0 con 0x7f5714004800 2024-04-12T19:51:30.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.945+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573015e070 con 0x7f5714004800 2024-04-12T19:51:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.946+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57300755c0 con 0x7f5714004800 2024-04-12T19:51:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.947+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f572001d460 con 0x7f5714004800 2024-04-12T19:51:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.948+0000 7f5726a00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 0x7f570c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.948+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f572008b830 con 0x7f5714004800 2024-04-12T19:51:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.948+0000 7f5727e00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 0x7f570c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.949+0000 7f5727e00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 0x7f570c06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5718036000 tx=0x7f571802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:30.952+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5720059fb0 con 0x7f5714004800 2024-04-12T19:51:31.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.232+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f57301664f0 con 0x7f5714004800 2024-04-12T19:51:31.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.238+0000 7f5726a00640 1 -- 172.21.15.32:0/1762355951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f57200598b0 con 0x7f5714004800 2024-04-12T19:51:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 msgr2=0x7f570c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 0x7f570c06edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5718036000 tx=0x7f571802e000 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 msgr2=0x7f573015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5720000f60 tx=0x7f572000ecd0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 shutdown_connections 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f570c06c8f0 0x7f570c06edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730157090 0x7f5730157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5730149580 0x7f5730146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 --2- 172.21.15.32:0/1762355951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5714004800 0x7f573015d360 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.240+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 >> 172.21.15.32:0/1762355951 conn(0x7f57300b23a0 msgr2=0x7f57300cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.241+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 shutdown_connections 2024-04-12T19:51:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.241+0000 7f572fe00640 1 -- 172.21.15.32:0/1762355951 wait complete. 2024-04-12T19:51:31.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-12T19:51:31.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.487+0000 7f6386000640 1 Processor -- start 2024-04-12T19:51:31.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.488+0000 7f6386000640 1 -- start start 2024-04-12T19:51:31.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.489+0000 7f6386000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6380162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.489+0000 7f6386000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6380162960 0x7f63800b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.489+0000 7f6386000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63800b3e90 con 0x7f6380148050 2024-04-12T19:51:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.489+0000 7f6386000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f63800afba0 con 0x7f6380162960 2024-04-12T19:51:31.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.489+0000 7f6386000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63800aed00 con 0x7f63800ae790 2024-04-12T19:51:31.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.490+0000 7f6384c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6380148050 0x7f63800b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56690/0 (socket says 172.21.15.32:56690) 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.490+0000 7f6384c00640 1 -- 172.21.15.32:0/2830103498 learned_addr learned my addr 172.21.15.32:0/2830103498 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.490+0000 7f637fe00640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6380162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.490+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> v2:172.21.15.169:3300/0 conn(0x7f6380162960 0x7f63800b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.490+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779164674 0 0) 0x7f63800b3e90 con 0x7f6380148050 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6368003520 con 0x7f6380148050 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2789539992 0 0) 0x7f6368003520 con 0x7f6380148050 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 msgr2=0x7f6380162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6380162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 >> v2:172.21.15.169:3300/0 conn(0x7f6380162960 msgr2=0x7f63800b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 --2- 172.21.15.32:0/2830103498 >> v2:172.21.15.169:3300/0 conn(0x7f6380162960 0x7f63800b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800aefe0 con 0x7f6380148050 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6374001670 con 0x7f6380148050 2024-04-12T19:51:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 >> v1:172.21.15.32:6789/0 conn(0x7f6380148050 legacy=0x7f63800b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63680039b0 0x7f6368003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6368004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63800afba0 con 0x7f6368004d40 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63800aed00 con 0x7f63680039b0 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f636800b9c0 con 0x7f63800ae790 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.491+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> v2:172.21.15.169:3300/0 conn(0x7f6380162960 0x7f63800b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:31.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f637fe00640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6368004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 -- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 msgr2=0x7f6368004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:31.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6384c00640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63680039b0 0x7f6368003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63800ae790 0x7f6368004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 -- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63680039b0 msgr2=0x7f6368003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63680039b0 0x7f6368003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6385600640 1 -- 172.21.15.32:0/2830103498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6368003520 con 0x7f6368004d40 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.492+0000 7f6384c00640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63680039b0 0x7f6368003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.494+0000 7f6385600640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63680051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f637800e350 tx=0x7f6368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af077b216c3b0db server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.494+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378009dc0 con 0x7f6368004d40 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.494+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637802cde0 con 0x7f6368004d40 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.494+0000 7f637f400640 1 -- 172.21.15.32:0/2830103498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378003b90 con 0x7f6368004d40 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 -- 172.21.15.32:0/2830103498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 msgr2=0x7f63680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 --2- 172.21.15.32:0/2830103498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63680051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f637800e350 tx=0x7f6368005830 comp rx=0 tx=0).stop 2024-04-12T19:51:31.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6385600640 1 -- 172.21.15.32:0/2830103498 reap_dead start 2024-04-12T19:51:31.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 -- 172.21.15.32:0/2830103498 shutdown_connections 2024-04-12T19:51:31.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 -- 172.21.15.32:0/2830103498 >> 172.21.15.32:0/2830103498 conn(0x7f63800b23a0 msgr2=0x7f63800b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:31.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 -- 172.21.15.32:0/2830103498 shutdown_connections 2024-04-12T19:51:31.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.495+0000 7f6386000640 1 -- 172.21.15.32:0/2830103498 wait complete. 2024-04-12T19:51:31.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.497+0000 7f6386000640 1 Processor -- start 2024-04-12T19:51:31.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.497+0000 7f6386000640 1 -- start start 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f638014ed00 0x7f63801f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f638014f180 0x7f6380142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6380148430 con 0x7f6368004d40 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63800afe40 con 0x7f638014f180 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6386000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63801485b0 con 0x7f638014ed00 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6384c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6384c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44682/0 (socket says 172.21.15.32:44682) 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.498+0000 7f6384c00640 1 -- 172.21.15.32:0/2496113281 learned_addr learned my addr 172.21.15.32:0/2496113281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f637fe00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f638014ed00 0x7f63801f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6385600640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f638014f180 0x7f6380142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 -- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f638014ed00 msgr2=0x7f63801f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f638014ed00 0x7f63801f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 -- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f638014f180 msgr2=0x7f6380142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f638014f180 0x7f6380142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 -- 172.21.15.32:0/2496113281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800ce570 con 0x7f6368004d40 2024-04-12T19:51:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6384c00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f637400d860 tx=0x7f637400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374019070 con 0x7f6368004d40 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63740074e0 con 0x7f6368004d40 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374004a20 con 0x7f6368004d40 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.499+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63800ccfb0 con 0x7f6368004d40 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.500+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63800cd570 con 0x7f6368004d40 2024-04-12T19:51:31.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.501+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63800755c0 con 0x7f6368004d40 2024-04-12T19:51:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.502+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6374004bc0 con 0x7f6368004d40 2024-04-12T19:51:31.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.503+0000 7f637cc00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 0x7f636006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:31.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.503+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f637408b330 con 0x7f6368004d40 2024-04-12T19:51:31.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.504+0000 7f637fe00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 0x7f636006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:31.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.504+0000 7f637fe00640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 0x7f636006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63801e83c0 tx=0x7f636c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:31.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.508+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6374059ad0 con 0x7f6368004d40 2024-04-12T19:51:31.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.775+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f63801452b0 con 0x7f6368004d40 2024-04-12T19:51:31.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.779+0000 7f637cc00640 1 -- 172.21.15.32:0/2496113281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f63740593d0 con 0x7f6368004d40 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 msgr2=0x7f636006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 0x7f636006ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63801e83c0 tx=0x7f636c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 msgr2=0x7f63800ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f637400d860 tx=0x7f637400dd20 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 shutdown_connections 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f636006c920 0x7f636006ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f638014f180 0x7f6380142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f638014ed00 0x7f63801f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 --2- 172.21.15.32:0/2496113281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6368004d40 0x7f63800ce030 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:31.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.781+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 >> 172.21.15.32:0/2496113281 conn(0x7f63800b23a0 msgr2=0x7f63801439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.782+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 shutdown_connections 2024-04-12T19:51:31.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:31.782+0000 7f6386000640 1 -- 172.21.15.32:0/2496113281 wait complete. 2024-04-12T19:51:31.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-12T19:51:32.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.006+0000 7f6af4c00640 1 Processor -- start 2024-04-12T19:51:32.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.006+0000 7f6af4c00640 1 -- start start 2024-04-12T19:51:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6af4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6af015d770 0x7f6af00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6af4c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6af00b59b0 0x7f6af00b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6af4c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af00cd940 con 0x7f6af00b0cb0 2024-04-12T19:51:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6af4c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6af00ce550 con 0x7f6af00b59b0 2024-04-12T19:51:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6af4c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6af00cdc20 con 0x7f6af015d770 2024-04-12T19:51:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6aef400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6af00b0cb0 0x7f6af00ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56704/0 (socket says 172.21.15.32:56704) 2024-04-12T19:51:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6aef400640 1 -- 172.21.15.32:0/807074520 learned_addr learned my addr 172.21.15.32:0/807074520 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:32.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.008+0000 7f6aefe00640 1 --2- 172.21.15.32:0/807074520 >> v2:172.21.15.32:3300/0 conn(0x7f6af00b59b0 0x7f6af00b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aeea00640 1 -- 172.21.15.32:0/807074520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1043009836 0 0) 0x7f6af00cd940 con 0x7f6af00b0cb0 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aeea00640 1 -- 172.21.15.32:0/807074520 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ad4003520 con 0x7f6af00b0cb0 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6ae7e00640 1 --2- 172.21.15.32:0/807074520 >> v2:172.21.15.169:3300/0 conn(0x7f6af015d770 0x7f6af00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aefe00640 1 -- 172.21.15.32:0/807074520 >> v2:172.21.15.169:3300/0 conn(0x7f6af015d770 msgr2=0x7f6af00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aefe00640 1 --2- 172.21.15.32:0/807074520 >> v2:172.21.15.169:3300/0 conn(0x7f6af015d770 0x7f6af00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aefe00640 1 -- 172.21.15.32:0/807074520 >> v1:172.21.15.32:6789/0 conn(0x7f6af00b0cb0 legacy=0x7f6af00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.009+0000 7f6aefe00640 1 -- 172.21.15.32:0/807074520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af00cdf00 con 0x7f6af00b59b0 2024-04-12T19:51:32.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.010+0000 7f6aefe00640 1 --2- 172.21.15.32:0/807074520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b59b0 0x7f6af00b5e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6af006c810 tx=0x7f6ae802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e83de496cdf975a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.010+0000 7f6aeea00640 1 -- 172.21.15.32:0/807074520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8039070 con 0x7f6af00b59b0 2024-04-12T19:51:32.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.010+0000 7f6aeea00640 1 -- 172.21.15.32:0/807074520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae802cc80 con 0x7f6af00b59b0 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.010+0000 7f6aeea00640 1 -- 172.21.15.32:0/807074520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8034640 con 0x7f6af00b59b0 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 -- 172.21.15.32:0/807074520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b59b0 msgr2=0x7f6af00b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 --2- 172.21.15.32:0/807074520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b59b0 0x7f6af00b5e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6af006c810 tx=0x7f6ae802bed0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 -- 172.21.15.32:0/807074520 shutdown_connections 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 --2- 172.21.15.32:0/807074520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b59b0 0x7f6af00b5e40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 --2- 172.21.15.32:0/807074520 >> v2:172.21.15.169:3300/0 conn(0x7f6af015d770 0x7f6af00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 -- 172.21.15.32:0/807074520 >> 172.21.15.32:0/807074520 conn(0x7f6af00b23a0 msgr2=0x7f6af00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 -- 172.21.15.32:0/807074520 shutdown_connections 2024-04-12T19:51:32.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.011+0000 7f6af4c00640 1 -- 172.21.15.32:0/807074520 wait complete. 2024-04-12T19:51:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.013+0000 7f6af4c00640 1 Processor -- start 2024-04-12T19:51:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.013+0000 7f6af4c00640 1 -- start start 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6af00b59b0 0x7f6af0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 0x7f6af0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6af00cd560 con 0x7f6af00b0cb0 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6af00b1090 con 0x7f6af015d770 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6af4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6af00cd6e0 con 0x7f6af00b59b0 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6aef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6aef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44694/0 (socket says 172.21.15.32:44694) 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.014+0000 7f6aef400640 1 -- 172.21.15.32:0/2022876295 learned_addr learned my addr 172.21.15.32:0/2022876295 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aefe00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 0x7f6af0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6ae7e00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6af00b59b0 0x7f6af0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 -- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6af00b59b0 msgr2=0x7f6af0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6af00b59b0 0x7f6af0155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 -- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 msgr2=0x7f6af0153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 0x7f6af0153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 -- 172.21.15.32:0/2022876295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af01565d0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aefe00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 0x7f6af0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6aef400640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c390 tx=0x7f6ae000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0017070 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae0005ce0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af01568b0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00072f0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.015+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af0156e70 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.017+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af00755c0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.018+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ae001a070 con 0x7f6af00b0cb0 2024-04-12T19:51:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.018+0000 7f6ae7400640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 0x7f6acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.019+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f6ae008ac30 con 0x7f6af00b0cb0 2024-04-12T19:51:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.019+0000 7f6ae7e00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 0x7f6acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.020+0000 7f6ae7e00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 0x7f6acc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6af00b5450 tx=0x7f6ad8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.023+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ae00593a0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.303+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f6af006ef40 con 0x7f6af00b0cb0 2024-04-12T19:51:32.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.307+0000 7f6ae7400640 1 -- 172.21.15.32:0/2022876295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f6ae0058ca0 con 0x7f6af00b0cb0 2024-04-12T19:51:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 msgr2=0x7f6acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 0x7f6acc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6af00b5450 tx=0x7f6ad8008040 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 msgr2=0x7f6af0144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c390 tx=0x7f6ae000c850 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 shutdown_connections 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6acc06c8f0 0x7f6acc06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6af015d770 0x7f6af0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6af00b59b0 0x7f6af0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 --2- 172.21.15.32:0/2022876295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6af00b0cb0 0x7f6af0144780 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 >> 172.21.15.32:0/2022876295 conn(0x7f6af00b23a0 msgr2=0x7f6af01599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.311+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 shutdown_connections 2024-04-12T19:51:32.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.312+0000 7f6af4c00640 1 -- 172.21.15.32:0/2022876295 wait complete. 2024-04-12T19:51:32.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-12T19:51:32.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.555+0000 7faaf7a00640 1 Processor -- start 2024-04-12T19:51:32.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.555+0000 7faaf7a00640 1 -- start start 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.556+0000 7faaf7a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf006c040 0x7faaf0073770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaf7a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf00d1730 con 0x7faaf015d930 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaf7a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf00d24b0 con 0x7faaf0146170 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaf7a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faaf00d2020 con 0x7faaf006c040 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaefe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf006c040 0x7faaf0073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaf5a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faaf015d930 0x7faaf015de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56706/0 (socket says 172.21.15.32:56706) 2024-04-12T19:51:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaf5a00640 1 -- 172.21.15.32:0/3029439724 learned_addr learned my addr 172.21.15.32:0/3029439724 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.557+0000 7faaefe00640 1 -- 172.21.15.32:0/3029439724 >> v1:172.21.15.169:6789/0 conn(0x7faaf0146170 legacy=0x7faaf006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.558+0000 7faaefe00640 1 -- 172.21.15.32:0/3029439724 >> v1:172.21.15.32:6789/0 conn(0x7faaf015d930 legacy=0x7faaf015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.558+0000 7faaefe00640 1 -- 172.21.15.32:0/3029439724 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf00d19d0 con 0x7faaf006c040 2024-04-12T19:51:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.558+0000 7faaef400640 1 -- 172.21.15.32:0/3029439724 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71581327 0 0) 0x7faaf00d1730 con 0x7faaf015d930 2024-04-12T19:51:32.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.559+0000 7faaefe00640 1 --2- 172.21.15.32:0/3029439724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf006c040 0x7faaf0073770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097a0 tx=0x7faadc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0638a214c8975b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.559+0000 7faaef400640 1 -- 172.21.15.32:0/3029439724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc038070 con 0x7faaf006c040 2024-04-12T19:51:32.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.559+0000 7faaef400640 1 -- 172.21.15.32:0/3029439724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc031700 con 0x7faaf006c040 2024-04-12T19:51:32.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.559+0000 7faaef400640 1 -- 172.21.15.32:0/3029439724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc031ad0 con 0x7faaf006c040 2024-04-12T19:51:32.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 -- 172.21.15.32:0/3029439724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf006c040 msgr2=0x7faaf0073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 --2- 172.21.15.32:0/3029439724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf006c040 0x7faaf0073770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097a0 tx=0x7faadc02b7a0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 -- 172.21.15.32:0/3029439724 shutdown_connections 2024-04-12T19:51:32.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 --2- 172.21.15.32:0/3029439724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf006c040 0x7faaf0073770 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 -- 172.21.15.32:0/3029439724 >> 172.21.15.32:0/3029439724 conn(0x7faaf00b6d70 msgr2=0x7faaf009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:32.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 -- 172.21.15.32:0/3029439724 shutdown_connections 2024-04-12T19:51:32.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.560+0000 7faaf7a00640 1 -- 172.21.15.32:0/3029439724 wait complete. 2024-04-12T19:51:32.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.562+0000 7faaf7a00640 1 Processor -- start 2024-04-12T19:51:32.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.562+0000 7faaf7a00640 1 -- start start 2024-04-12T19:51:32.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf006c040 0x7faaf013c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf015d930 0x7faaf013d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0146550 con 0x7faaf0146170 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faaf00ae810 con 0x7faaf006c040 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faaf015e510 con 0x7faaf015d930 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44710/0 (socket says 172.21.15.32:44710) 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 -- 172.21.15.32:0/1409154440 learned_addr learned my addr 172.21.15.32:0/1409154440 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:32.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaf5000640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf006c040 0x7faaf013c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 -- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf015d930 msgr2=0x7faaf013d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf015d930 0x7faaf013d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 -- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf006c040 msgr2=0x7faaf013c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf006c040 0x7faaf013c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.563+0000 7faaefe00640 1 -- 172.21.15.32:0/1409154440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf015d6a0 con 0x7faaf0146170 2024-04-12T19:51:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaefe00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097a0 tx=0x7faadc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc043070 con 0x7faaf0146170 2024-04-12T19:51:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc033a30 con 0x7faaf0146170 2024-04-12T19:51:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc038040 con 0x7faaf0146170 2024-04-12T19:51:32.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf013ca80 con 0x7faaf0146170 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.564+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf009e850 con 0x7faaf0146170 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.566+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0068c80 con 0x7faaf0146170 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.567+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faadc046070 con 0x7faaf0146170 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.568+0000 7faaecc00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.568+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7faadc0ad920 con 0x7faaf0146170 2024-04-12T19:51:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.568+0000 7faaf5000640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.569+0000 7faaf5000640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faae0004ea0 tx=0x7faae0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:32.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.571+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faadc030070 con 0x7faaf0146170 2024-04-12T19:51:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.850+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7faaf015f7f0 con 0x7faaf0146170 2024-04-12T19:51:32.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.855+0000 7faaecc00640 1 -- 172.21.15.32:0/1409154440 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7faadc07c110 con 0x7faaf0146170 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 msgr2=0x7faad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 0x7faad006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faae0004ea0 tx=0x7faae0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 msgr2=0x7faaf0144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faadc0097a0 tx=0x7faadc0046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 shutdown_connections 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7faad006c8f0 0x7faad006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf015d930 0x7faaf013d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf0146170 0x7faaf0144060 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.858+0000 7faaf7a00640 1 --2- 172.21.15.32:0/1409154440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf006c040 0x7faaf013c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.859+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 >> 172.21.15.32:0/1409154440 conn(0x7faaf00b6d70 msgr2=0x7faaf015e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.859+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 shutdown_connections 2024-04-12T19:51:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:32.859+0000 7faaf7a00640 1 -- 172.21.15.32:0/1409154440 wait complete. 2024-04-12T19:51:32.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.105+0000 7fe196a00640 1 Processor -- start 2024-04-12T19:51:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.105+0000 7fe196a00640 1 -- start start 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe190072660 0x7fe19006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe19015cbe0 0x7fe19015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19015dbb0 con 0x7fe1900cee00 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe19006f4f0 con 0x7fe190072660 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe19015d8d0 con 0x7fe19015cbe0 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe194c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe190072660 0x7fe19006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe194c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe190072660 0x7fe19006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44714/0 (socket says 172.21.15.32:44714) 2024-04-12T19:51:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe194c00640 1 -- 172.21.15.32:0/961980966 learned_addr learned my addr 172.21.15.32:0/961980966 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.107+0000 7fe196000640 1 --2- 172.21.15.32:0/961980966 >> v2:172.21.15.169:3300/0 conn(0x7fe19015cbe0 0x7fe19015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.108+0000 7fe194c00640 1 -- 172.21.15.32:0/961980966 >> v2:172.21.15.169:3300/0 conn(0x7fe19015cbe0 msgr2=0x7fe19015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.108+0000 7fe194c00640 1 --2- 172.21.15.32:0/961980966 >> v2:172.21.15.169:3300/0 conn(0x7fe19015cbe0 0x7fe19015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.108+0000 7fe194c00640 1 -- 172.21.15.32:0/961980966 >> v1:172.21.15.169:6789/0 conn(0x7fe1900cee00 legacy=0x7fe190071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.108+0000 7fe194c00640 1 -- 172.21.15.32:0/961980966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19006dde0 con 0x7fe190072660 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.109+0000 7fe194c00640 1 --2- 172.21.15.32:0/961980966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19006e8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1900b02a0 tx=0x7fe17c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e501a3c8094c6cda server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.109+0000 7fe18fe00640 1 -- 172.21.15.32:0/961980966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c035020 con 0x7fe190072660 2024-04-12T19:51:33.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.109+0000 7fe18fe00640 1 -- 172.21.15.32:0/961980966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17c031870 con 0x7fe190072660 2024-04-12T19:51:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.109+0000 7fe18fe00640 1 -- 172.21.15.32:0/961980966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c031c40 con 0x7fe190072660 2024-04-12T19:51:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 -- 172.21.15.32:0/961980966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 msgr2=0x7fe19006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 --2- 172.21.15.32:0/961980966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19006e8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1900b02a0 tx=0x7fe17c02b910 comp rx=0 tx=0).stop 2024-04-12T19:51:33.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 -- 172.21.15.32:0/961980966 shutdown_connections 2024-04-12T19:51:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 --2- 172.21.15.32:0/961980966 >> v2:172.21.15.169:3300/0 conn(0x7fe19015cbe0 0x7fe19015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 --2- 172.21.15.32:0/961980966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19006e8c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 -- 172.21.15.32:0/961980966 >> 172.21.15.32:0/961980966 conn(0x7fe19006b930 msgr2=0x7fe19006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 -- 172.21.15.32:0/961980966 shutdown_connections 2024-04-12T19:51:33.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.110+0000 7fe196a00640 1 -- 172.21.15.32:0/961980966 wait complete. 2024-04-12T19:51:33.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.112+0000 7fe196a00640 1 Processor -- start 2024-04-12T19:51:33.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.112+0000 7fe196a00640 1 -- start start 2024-04-12T19:51:33.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1900cee00 0x7fe1901554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19015cbe0 0x7fe190155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe19015d5b0 con 0x7fe190072660 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe19006e3a0 con 0x7fe1900cee00 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.113+0000 7fe196a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe19015dda0 con 0x7fe19015cbe0 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44716/0 (socket says 172.21.15.32:44716) 2024-04-12T19:51:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 -- 172.21.15.32:0/2363538060 learned_addr learned my addr 172.21.15.32:0/2363538060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe194c00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1900cee00 0x7fe1901554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 -- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19015cbe0 msgr2=0x7fe190155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe196000640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19015cbe0 0x7fe190155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19015cbe0 0x7fe190155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 -- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1900cee00 msgr2=0x7fe1901554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1900cee00 0x7fe1901554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.114+0000 7fe195600640 1 -- 172.21.15.32:0/2363538060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1901f5530 con 0x7fe190072660 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe195600640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c430 tx=0x7fe18400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184011790 con 0x7fe190072660 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe184017070 con 0x7fe190072660 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184011bb0 con 0x7fe190072660 2024-04-12T19:51:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1901f5250 con 0x7fe190072660 2024-04-12T19:51:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.115+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1900b3d60 con 0x7fe190072660 2024-04-12T19:51:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.116+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1900755c0 con 0x7fe190072660 2024-04-12T19:51:33.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.117+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe18401c460 con 0x7fe190072660 2024-04-12T19:51:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.118+0000 7fe18d600640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.118+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fe18408b740 con 0x7fe190072660 2024-04-12T19:51:33.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.118+0000 7fe194c00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.119+0000 7fe194c00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 0x7fe17006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe17c036000 tx=0x7fe17c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.123+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe184059e20 con 0x7fe190072660 2024-04-12T19:51:33.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.331+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe190157a90 con 0x7fe190072660 2024-04-12T19:51:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.333+0000 7fe18d600640 1 -- 172.21.15.32:0/2363538060 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+12879 (secure 0 0 0) 0x7fe184059720 con 0x7fe190072660 2024-04-12T19:51:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:33.333 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:29.103286+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.335+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 msgr2=0x7fe17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.335+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 0x7fe17006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe17c036000 tx=0x7fe17c03c040 comp rx=0 tx=0).stop 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.335+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 msgr2=0x7fe19014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.335+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c430 tx=0x7fe18400c8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 shutdown_connections 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe19015cbe0 0x7fe190155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1900cee00 0x7fe1901554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 --2- 172.21.15.32:0/2363538060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe190072660 0x7fe19014ce50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 >> 172.21.15.32:0/2363538060 conn(0x7fe19006b930 msgr2=0x7fe19006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 shutdown_connections 2024-04-12T19:51:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.336+0000 7fe196a00640 1 -- 172.21.15.32:0/2363538060 wait complete. 2024-04-12T19:51:33.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:33.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.588+0000 7f38cf200640 1 Processor -- start 2024-04-12T19:51:33.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.588+0000 7f38cf200640 1 -- start start 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.589+0000 7f38cf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f38d00d26b0 0x7f38d00b7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.589+0000 7f38cf200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d00cfef0 con 0x7f38d00b7e80 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.589+0000 7f38cf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d00d2bb0 con 0x7f38d0161f80 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.589+0000 7f38cf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f38d00d2e90 con 0x7f38d00d26b0 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.590+0000 7f38ce800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38d00b7e80 0x7f38d00d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56712/0 (socket says 172.21.15.32:56712) 2024-04-12T19:51:33.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.590+0000 7f38ce800640 1 -- 172.21.15.32:0/2095814078 learned_addr learned my addr 172.21.15.32:0/2095814078 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.590+0000 7f38cd400640 1 --2- 172.21.15.32:0/2095814078 >> v2:172.21.15.169:3300/0 conn(0x7f38d00d26b0 0x7f38d00b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.590+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803645174 0 0) 0x7f38d00cfef0 con 0x7f38d00b7e80 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.590+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b4003520 con 0x7f38d00b7e80 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1066780078 0 0) 0x7f38b4003520 con 0x7f38d00b7e80 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 >> v2:172.21.15.169:3300/0 conn(0x7f38d00d26b0 msgr2=0x7f38d00b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 --2- 172.21.15.32:0/2095814078 >> v2:172.21.15.169:3300/0 conn(0x7f38d00d26b0 0x7f38d00b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 >> v1:172.21.15.169:6789/0 conn(0x7f38d0161f80 legacy=0x7f38d00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00d3170 con 0x7f38d00b7e80 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f38c80019f0 con 0x7f38d00b7e80 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 >> v1:172.21.15.32:6789/0 conn(0x7f38d00b7e80 legacy=0x7f38d00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38b4003a00 0x7f38b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b4004800 0x7f38b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 0x7f38b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00d2e90 con 0x7f38b40051c0 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38b400c0c0 con 0x7f38b4003a00 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38b400c2c0 con 0x7f38b4004800 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.591+0000 7f38ce800640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 0x7f38b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38cde00640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38b4003a00 0x7f38b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38cd400640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b4004800 0x7f38b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38ce800640 1 -- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b4004800 msgr2=0x7f38b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38ce800640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b4004800 0x7f38b4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38ce800640 1 -- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38b4003a00 msgr2=0x7f38b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38ce800640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38b4003a00 0x7f38b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.592+0000 7f38ce800640 1 -- 172.21.15.32:0/2095814078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b4003520 con 0x7f38b40051c0 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.593+0000 7f38ce800640 1 -- 172.21.15.32:0/2095814078 reap_dead start 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.593+0000 7f38ce800640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 0x7f38b400b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a7b0 tx=0x7f38b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d96698385a1af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.593+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800a3d0 con 0x7f38b40051c0 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.593+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8034650 con 0x7f38b40051c0 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.593+0000 7f38cca00640 1 -- 172.21.15.32:0/2095814078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8034a40 con 0x7f38b40051c0 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 -- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 msgr2=0x7f38b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 0x7f38b400b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38c800a7b0 tx=0x7f38b4005830 comp rx=0 tx=0).stop 2024-04-12T19:51:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 -- 172.21.15.32:0/2095814078 shutdown_connections 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 --2- 172.21.15.32:0/2095814078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38b40051c0 0x7f38b400b9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 -- 172.21.15.32:0/2095814078 >> 172.21.15.32:0/2095814078 conn(0x7f38d006ce40 msgr2=0x7f38d006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 -- 172.21.15.32:0/2095814078 shutdown_connections 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.594+0000 7f38cf200640 1 -- 172.21.15.32:0/2095814078 wait complete. 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.596+0000 7f38cf200640 1 Processor -- start 2024-04-12T19:51:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.596+0000 7f38cf200640 1 -- start start 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b40051c0 0x7f38d01f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0154290 0x7f38d01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00b7940 con 0x7f38d00c9660 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0162360 con 0x7f38d0154290 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38d00b7ac0 con 0x7f38b40051c0 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44722/0 (socket says 172.21.15.32:44722) 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38cd400640 1 -- 172.21.15.32:0/717252393 learned_addr learned my addr 172.21.15.32:0/717252393 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.597+0000 7f38ce800640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0154290 0x7f38d01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 -- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b40051c0 msgr2=0x7f38d01f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b40051c0 0x7f38d01f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 -- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0154290 msgr2=0x7f38d01445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0154290 0x7f38d01445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 -- 172.21.15.32:0/717252393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00c9bd0 con 0x7f38d00c9660 2024-04-12T19:51:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cd400640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0108c0 tx=0x7f38bc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00f040 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc00eb20 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d00c9eb0 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc01c400 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.598+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d00ca470 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.600+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d00755c0 con 0x7f38d00c9660 2024-04-12T19:51:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.600+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38bc0253f0 con 0x7f38d00c9660 2024-04-12T19:51:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.601+0000 7f38c6000640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 0x7f38b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.602+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f38bc08bf00 con 0x7f38d00c9660 2024-04-12T19:51:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.602+0000 7f38cde00640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 0x7f38b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.603+0000 7f38cde00640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 0x7f38b006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f38c0007fd0 tx=0x7f38c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.606+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38bc018030 con 0x7f38d00c9660 2024-04-12T19:51:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.810+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38d00d2490 con 0x7f38d00c9660 2024-04-12T19:51:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.810+0000 7f38c6000640 1 -- 172.21.15.32:0/717252393 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f38bc05a670 con 0x7f38d00c9660 2024-04-12T19:51:33.810 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 msgr2=0x7f38b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 0x7f38b006ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f38c0007fd0 tx=0x7f38c0009040 comp rx=0 tx=0).stop 2024-04-12T19:51:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 msgr2=0x7f38d015a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:33.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0108c0 tx=0x7f38bc010d80 comp rx=0 tx=0).stop 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 shutdown_connections 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f38b006c920 0x7f38b006ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0154290 0x7f38d01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38d00c9660 0x7f38d015a520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 --2- 172.21.15.32:0/717252393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38b40051c0 0x7f38d01f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 >> 172.21.15.32:0/717252393 conn(0x7f38d006ce40 msgr2=0x7f38d015d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 shutdown_connections 2024-04-12T19:51:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:33.813+0000 7f38cf200640 1 -- 172.21.15.32:0/717252393 wait complete. 2024-04-12T19:51:33.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-12T19:51:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.048+0000 7fb9c3200640 1 Processor -- start 2024-04-12T19:51:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.048+0000 7fb9c3200640 1 -- start start 2024-04-12T19:51:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9bc0cedf0 0x7fb9bc0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb9bc0a9640 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9bc0cf250 con 0x7fb9bc0cedf0 2024-04-12T19:51:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.049+0000 7fb9c3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc0b4be0 con 0x7fb9bc1461f0 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52852/0 (socket says 172.21.15.32:52852) 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 -- 172.21.15.32:0/1529357891 learned_addr learned my addr 172.21.15.32:0/1529357891 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9bbe00640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9b3400640 1 --2- 172.21.15.32:0/1529357891 >> v2:172.21.15.169:3300/0 conn(0x7fb9bc0cedf0 0x7fb9bc0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 -- 172.21.15.32:0/1529357891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 msgr2=0x7fb9bc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 -- 172.21.15.32:0/1529357891 >> v2:172.21.15.169:3300/0 conn(0x7fb9bc0cedf0 msgr2=0x7fb9bc0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 --2- 172.21.15.32:0/1529357891 >> v2:172.21.15.169:3300/0 conn(0x7fb9bc0cedf0 0x7fb9bc0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.050+0000 7fb9c1200640 1 -- 172.21.15.32:0/1529357891 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc0b4620 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9c1200640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30830ce115799119 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9bb400640 1 -- 172.21.15.32:0/1529357891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4035020 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9bb400640 1 -- 172.21.15.32:0/1529357891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4031640 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9bb400640 1 -- 172.21.15.32:0/1529357891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b402d040 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9c3200640 1 -- 172.21.15.32:0/1529357891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 msgr2=0x7fb9bc0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.052+0000 7fb9c3200640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 -- 172.21.15.32:0/1529357891 shutdown_connections 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc0cd2b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 --2- 172.21.15.32:0/1529357891 >> v2:172.21.15.169:3300/0 conn(0x7fb9bc0cedf0 0x7fb9bc0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 --2- 172.21.15.32:0/1529357891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 -- 172.21.15.32:0/1529357891 >> 172.21.15.32:0/1529357891 conn(0x7fb9bc0b27b0 msgr2=0x7fb9bc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 -- 172.21.15.32:0/1529357891 shutdown_connections 2024-04-12T19:51:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.053+0000 7fb9c3200640 1 -- 172.21.15.32:0/1529357891 wait complete. 2024-04-12T19:51:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.054+0000 7fb9c3200640 1 Processor -- start 2024-04-12T19:51:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.055+0000 7fb9c3200640 1 -- start start 2024-04-12T19:51:34.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 0x7fb9bc1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc15d290 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc15cfb0 con 0x7fb9bc0cedf0 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc1465d0 con 0x7fb9bc1461f0 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52856/0 (socket says 172.21.15.32:52856) 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 -- 172.21.15.32:0/3583963310 learned_addr learned my addr 172.21.15.32:0/3583963310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9b3400640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 0x7fb9bc1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 -- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 msgr2=0x7fb9bc13ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9c1200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 -- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 msgr2=0x7fb9bc1419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 0x7fb9bc1419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9bbe00640 1 -- 172.21.15.32:0/3583963310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc0afbc0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.056+0000 7fb9b3400640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 0x7fb9bc1419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9bbe00640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00d5f0 tx=0x7fb9ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0125f0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac019070 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0129d0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc1421f0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.057+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1427b0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.058+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc0d0000 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.059+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9ac012b70 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.061+0000 7fb9b8c00640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 0x7fb99806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.061+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fb9ac08bc50 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.061+0000 7fb9b3400640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 0x7fb99806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.062+0000 7fb9b3400640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 0x7fb99806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc15c460 tx=0x7fb9a400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.066+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9bc0d0000 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.279+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb9bc068c80 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.280+0000 7fb9b8c00640 1 -- 172.21.15.32:0/3583963310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fb9bc0afbc0 con 0x7fb9bc0b3e10 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-12T19:51:34.280 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-12T19:51:34.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:34.281 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.282+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 msgr2=0x7fb99806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.282+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 0x7fb99806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc15c460 tx=0x7fb9a400b040 comp rx=0 tx=0).stop 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.282+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 msgr2=0x7fb9bc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.282+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00d5f0 tx=0x7fb9ac00dab0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 shutdown_connections 2024-04-12T19:51:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb99806c920 0x7fb99806ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9bc1461f0 0x7fb9bc13ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9bc0cedf0 0x7fb9bc1419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 --2- 172.21.15.32:0/3583963310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9bc0b3e10 0x7fb9bc15c4b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 >> 172.21.15.32:0/3583963310 conn(0x7fb9bc0b27b0 msgr2=0x7fb9bc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 shutdown_connections 2024-04-12T19:51:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.283+0000 7fb9c3200640 1 -- 172.21.15.32:0/3583963310 wait complete. 2024-04-12T19:51:34.303 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-12T19:51:34.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-12T19:51:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.535+0000 7f67fa600640 1 Processor -- start 2024-04-12T19:51:34.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.535+0000 7f67fa600640 1 -- start start 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.536+0000 7f67fa600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f40b6b80 0x7f67f40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.536+0000 7f67fa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40b78e0 0x7f67f40b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67fa600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f40cfe30 con 0x7f67f40cf510 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67fa600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f67f40d3f60 con 0x7f67f40b6b80 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67fa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f67f40d3350 con 0x7f67f40b78e0 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67f3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f40b6b80 0x7f67f40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67f3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f40b6b80 0x7f67f40cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52870/0 (socket says 172.21.15.32:52870) 2024-04-12T19:51:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67f3e00640 1 -- 172.21.15.32:0/770619336 learned_addr learned my addr 172.21.15.32:0/770619336 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67f3e00640 1 -- 172.21.15.32:0/770619336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40b78e0 msgr2=0x7f67f40b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.537+0000 7f67f3e00640 1 --2- 172.21.15.32:0/770619336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40b78e0 0x7f67f40b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.538+0000 7f67f3e00640 1 -- 172.21.15.32:0/770619336 >> v1:172.21.15.169:6789/0 conn(0x7f67f40cf510 legacy=0x7f67f40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.538+0000 7f67f3e00640 1 -- 172.21.15.32:0/770619336 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f40d3630 con 0x7f67f40b6b80 2024-04-12T19:51:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.539+0000 7f67f3e00640 1 --2- 172.21.15.32:0/770619336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f40cf8f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009870 tx=0x7f67e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996aab0fee74ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.539+0000 7f67f3400640 1 -- 172.21.15.32:0/770619336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0034070 con 0x7f67f40b6b80 2024-04-12T19:51:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.539+0000 7f67f3400640 1 -- 172.21.15.32:0/770619336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e0030840 con 0x7f67f40b6b80 2024-04-12T19:51:34.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.539+0000 7f67f3400640 1 -- 172.21.15.32:0/770619336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0030c10 con 0x7f67f40b6b80 2024-04-12T19:51:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 -- 172.21.15.32:0/770619336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 msgr2=0x7f67f40cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 --2- 172.21.15.32:0/770619336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f40cf8f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009870 tx=0x7f67e002b980 comp rx=0 tx=0).stop 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 -- 172.21.15.32:0/770619336 shutdown_connections 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 --2- 172.21.15.32:0/770619336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40b78e0 0x7f67f40b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 --2- 172.21.15.32:0/770619336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f40cf8f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 -- 172.21.15.32:0/770619336 >> 172.21.15.32:0/770619336 conn(0x7f67f40d13a0 msgr2=0x7f67f40d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 -- 172.21.15.32:0/770619336 shutdown_connections 2024-04-12T19:51:34.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.540+0000 7f67fa600640 1 -- 172.21.15.32:0/770619336 wait complete. 2024-04-12T19:51:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.542+0000 7f67fa600640 1 Processor -- start 2024-04-12T19:51:34.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.542+0000 7f67fa600640 1 -- start start 2024-04-12T19:51:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f40b78e0 0x7f67f414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40cf510 0x7f67f4161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67f40d43f0 con 0x7f67f40b6b80 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67f40d3bf0 con 0x7f67f40b78e0 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.543+0000 7f67fa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f67f40d4570 con 0x7f67f40cf510 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52882/0 (socket says 172.21.15.32:52882) 2024-04-12T19:51:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 -- 172.21.15.32:0/4170875109 learned_addr learned my addr 172.21.15.32:0/4170875109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 -- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40cf510 msgr2=0x7f67f4161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40cf510 0x7f67f4161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 -- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f40b78e0 msgr2=0x7f67f414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f40b78e0 0x7f67f414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 -- 172.21.15.32:0/4170875109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f41622f0 con 0x7f67f40b6b80 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.544+0000 7f67f9200640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c450 tx=0x7f67e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.545+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40052d0 con 0x7f67f40b6b80 2024-04-12T19:51:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.545+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e4005470 con 0x7f67f40b6b80 2024-04-12T19:51:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.545+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4012870 con 0x7f67f40b6b80 2024-04-12T19:51:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.545+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f415a270 con 0x7f67f40b6b80 2024-04-12T19:51:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.545+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f41584c0 con 0x7f67f40b6b80 2024-04-12T19:51:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.546+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f4068b40 con 0x7f67f40b6b80 2024-04-12T19:51:34.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.547+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67e4012a10 con 0x7f67f40b6b80 2024-04-12T19:51:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.548+0000 7f67f0c00640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 0x7f67d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.548+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f67e408b400 con 0x7f67f40b6b80 2024-04-12T19:51:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.548+0000 7f67f3e00640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 0x7f67d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.549+0000 7f67f3e00640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 0x7f67d406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67e0036000 tx=0x7f67e0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.553+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67e4059b70 con 0x7f67f40b6b80 2024-04-12T19:51:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:34.757+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f67f415a860 con 0x7f67f40b6b80 2024-04-12T19:51:35.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.270+0000 7f67f0c00640 1 -- 172.21.15.32:0/4170875109 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f67e4059470 con 0x7f67f40b6b80 2024-04-12T19:51:35.270 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-12T19:51:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 msgr2=0x7f67d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 0x7f67d406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67e0036000 tx=0x7f67e0030000 comp rx=0 tx=0).stop 2024-04-12T19:51:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 msgr2=0x7f67f415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c450 tx=0x7f67e400c910 comp rx=0 tx=0).stop 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 shutdown_connections 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f67d406c920 0x7f67d406ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67f40cf510 0x7f67f4161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f40b78e0 0x7f67f414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 --2- 172.21.15.32:0/4170875109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f40b6b80 0x7f67f415aa80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 >> 172.21.15.32:0/4170875109 conn(0x7f67f40d13a0 msgr2=0x7f67f40d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 shutdown_connections 2024-04-12T19:51:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.274+0000 7f67fa600640 1 -- 172.21.15.32:0/4170875109 wait complete. 2024-04-12T19:51:35.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-12T19:51:35.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.529+0000 7f9322600640 1 Processor -- start 2024-04-12T19:51:35.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.529+0000 7f9322600640 1 -- start start 2024-04-12T19:51:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9322600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f931c071f50 0x7f931c0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9322600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f931c0b47a0 0x7f931c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9322600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f931c0b3720 con 0x7f931c1483f0 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9322600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f931c0b39e0 con 0x7f931c071f50 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9322600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f931c0b3cc0 con 0x7f931c0b47a0 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.531+0000 7f9313e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f931c071f50 0x7f931c0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f931c071f50 0x7f931c0723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52884/0 (socket says 172.21.15.32:52884) 2024-04-12T19:51:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 -- 172.21.15.32:0/1273855558 learned_addr learned my addr 172.21.15.32:0/1273855558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 -- 172.21.15.32:0/1273855558 >> v2:172.21.15.169:3300/0 conn(0x7f931c0b47a0 msgr2=0x7f931c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9321c00640 1 --2- 172.21.15.32:0/1273855558 >> v2:172.21.15.169:3300/0 conn(0x7f931c0b47a0 0x7f931c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 --2- 172.21.15.32:0/1273855558 >> v2:172.21.15.169:3300/0 conn(0x7f931c0b47a0 0x7f931c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 -- 172.21.15.32:0/1273855558 >> v1:172.21.15.169:6789/0 conn(0x7f931c1483f0 legacy=0x7f931c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.532+0000 7f9313e00640 1 -- 172.21.15.32:0/1273855558 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c0b3fa0 con 0x7f931c071f50 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.533+0000 7f9313e00640 1 --2- 172.21.15.32:0/1273855558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c0723e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f931c0ad940 tx=0x7f930402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5130c4d36cd27984 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f931be00640 1 -- 172.21.15.32:0/1273855558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304035020 con 0x7f931c071f50 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f931be00640 1 -- 172.21.15.32:0/1273855558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9304031870 con 0x7f931c071f50 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f931be00640 1 -- 172.21.15.32:0/1273855558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304031c40 con 0x7f931c071f50 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 -- 172.21.15.32:0/1273855558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 msgr2=0x7f931c0723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 --2- 172.21.15.32:0/1273855558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c0723e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f931c0ad940 tx=0x7f930402b910 comp rx=0 tx=0).stop 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 -- 172.21.15.32:0/1273855558 shutdown_connections 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 --2- 172.21.15.32:0/1273855558 >> v2:172.21.15.169:3300/0 conn(0x7f931c0b47a0 0x7f931c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 --2- 172.21.15.32:0/1273855558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c0723e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.534+0000 7f9322600640 1 -- 172.21.15.32:0/1273855558 >> 172.21.15.32:0/1273855558 conn(0x7f931c06b930 msgr2=0x7f931c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.535+0000 7f9322600640 1 -- 172.21.15.32:0/1273855558 shutdown_connections 2024-04-12T19:51:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.535+0000 7f9322600640 1 -- 172.21.15.32:0/1273855558 wait complete. 2024-04-12T19:51:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.536+0000 7f9322600640 1 Processor -- start 2024-04-12T19:51:35.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.536+0000 7f9322600640 1 -- start start 2024-04-12T19:51:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931c0b47a0 0x7f931c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c1483f0 0x7f931c13c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f931c0afea0 con 0x7f931c071f50 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f931c06f090 con 0x7f931c1483f0 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.537+0000 7f9322600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f931c0b0020 con 0x7f931c0b47a0 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52900/0 (socket says 172.21.15.32:52900) 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 -- 172.21.15.32:0/92957227 learned_addr learned my addr 172.21.15.32:0/92957227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321c00640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c1483f0 0x7f931c13c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9313e00640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931c0b47a0 0x7f931c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 -- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931c0b47a0 msgr2=0x7f931c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931c0b47a0 0x7f931c0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 -- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c1483f0 msgr2=0x7f931c13c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c1483f0 0x7f931c13c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.538+0000 7f9321200640 1 -- 172.21.15.32:0/92957227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c0ceda0 con 0x7f931c071f50 2024-04-12T19:51:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9321200640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f930c00c410 tx=0x7f930c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c017070 con 0x7f931c071f50 2024-04-12T19:51:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c0035f0 con 0x7f931c071f50 2024-04-12T19:51:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c003a20 con 0x7f931c071f50 2024-04-12T19:51:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c0cf080 con 0x7f931c071f50 2024-04-12T19:51:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.539+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c0cf640 con 0x7f931c071f50 2024-04-12T19:51:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.540+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931c0755c0 con 0x7f931c071f50 2024-04-12T19:51:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.542+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f930c01a070 con 0x7f931c071f50 2024-04-12T19:51:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.543+0000 7f9319600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 0x7f92f806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.543+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f930c08b300 con 0x7f931c071f50 2024-04-12T19:51:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.543+0000 7f9313e00640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 0x7f92f806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.544+0000 7f9313e00640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 0x7f92f806ee00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9304036000 tx=0x7f930403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.547+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f930c021020 con 0x7f931c071f50 2024-04-12T19:51:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:35.747+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f931c072940 con 0x7f931c071f50 2024-04-12T19:51:36.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.287+0000 7f9319600640 1 -- 172.21.15.32:0/92957227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f930c0598f0 con 0x7f931c071f50 2024-04-12T19:51:36.288 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-12T19:51:36.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.292+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 msgr2=0x7f92f806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.292+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 0x7f92f806ee00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9304036000 tx=0x7f930403c040 comp rx=0 tx=0).stop 2024-04-12T19:51:36.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.292+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 msgr2=0x7f931c1429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.292+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f930c00c410 tx=0x7f930c00c8d0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 shutdown_connections 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f92f806c940 0x7f92f806ee00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c1483f0 0x7f931c13c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f931c0b47a0 0x7f931c0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 --2- 172.21.15.32:0/92957227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f931c071f50 0x7f931c1429e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.294+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 >> 172.21.15.32:0/92957227 conn(0x7f931c06b930 msgr2=0x7f931c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:36.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.299+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 shutdown_connections 2024-04-12T19:51:36.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.299+0000 7f9322600640 1 -- 172.21.15.32:0/92957227 wait complete. 2024-04-12T19:51:36.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-12T19:51:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.516+0000 7fd6bdc00640 1 Processor -- start 2024-04-12T19:51:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.516+0000 7fd6bdc00640 1 -- start start 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.518+0000 7fd6bdc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd6b8146240 0x7fd6b80b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.518+0000 7fd6bdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 0x7fd6b80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.518+0000 7fd6bdc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b806e660 con 0x7fd6b80721a0 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.518+0000 7fd6bdc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b8072d40 con 0x7fd6b8146240 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.518+0000 7fd6bdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8146620 con 0x7fd6b806f420 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6bd200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6b80721a0 0x7fd6b8072630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54404/0 (socket says 172.21.15.32:54404) 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6bd200640 1 -- 172.21.15.32:0/3130331752 learned_addr learned my addr 172.21.15.32:0/3130331752 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6b7400640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 0x7fd6b80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3130331752 >> v2:172.21.15.169:3300/0 conn(0x7fd6b8146240 0x7fd6b80b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614243438 0 0) 0x7fd6b806e660 con 0x7fd6b80721a0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.519+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd69c003520 con 0x7fd6b80721a0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 881522364 0 0) 0x7fd69c003520 con 0x7fd6b80721a0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 msgr2=0x7fd6b80b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 0x7fd6b80b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 >> v2:172.21.15.169:3300/0 conn(0x7fd6b8146240 msgr2=0x7fd6b80b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 --2- 172.21.15.32:0/3130331752 >> v2:172.21.15.169:3300/0 conn(0x7fd6b8146240 0x7fd6b80b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b806e940 con 0x7fd6b80721a0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fd6b00019f0 con 0x7fd6b80721a0 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 >> v1:172.21.15.32:6789/0 conn(0x7fd6b80721a0 legacy=0x7fd6b8072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 0x7fd69c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8072d40 con 0x7fd69c0039b0 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8146620 con 0x7fd69c004800 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c00b9c0 con 0x7fd6b806f420 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 msgr2=0x7fd69c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.520+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b806f420 0x7fd69c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.521+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c004800 msgr2=0x7fd69c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.521+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c004800 0x7fd69c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.521+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3130331752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c003520 con 0x7fd69c0039b0 2024-04-12T19:51:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009ac0 tx=0x7fd69c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665735c9b5bd59f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8039070 con 0x7fd69c0039b0 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a802c710 con 0x7fd69c0039b0 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6b6a00640 1 -- 172.21.15.32:0/3130331752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8009690 con 0x7fd69c0039b0 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3130331752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 msgr2=0x7fd69c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3130331752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd69c003dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009ac0 tx=0x7fd69c005830 comp rx=0 tx=0).stop 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6bd200640 1 -- 172.21.15.32:0/3130331752 reap_dead start 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3130331752 shutdown_connections 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.522+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3130331752 >> 172.21.15.32:0/3130331752 conn(0x7fd6b806b930 msgr2=0x7fd6b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.523+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3130331752 shutdown_connections 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.523+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3130331752 wait complete. 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.524+0000 7fd6bdc00640 1 Processor -- start 2024-04-12T19:51:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.524+0000 7fd6bdc00640 1 -- start start 2024-04-12T19:51:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b815e150 0x7fd6b815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b815ed30 0x7fd6b815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80b2a60 con 0x7fd69c0039b0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b806ef00 con 0x7fd6b815ed30 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.525+0000 7fd6bdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80b2be0 con 0x7fd6b815e150 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52918/0 (socket says 172.21.15.32:52918) 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3362028831 learned_addr learned my addr 172.21.15.32:0/3362028831 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b815e150 msgr2=0x7fd6b815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b815e150 0x7fd6b815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b815ed30 msgr2=0x7fd6b815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b815ed30 0x7fd6b815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 -- 172.21.15.32:0/3362028831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b80b5a00 con 0x7fd69c0039b0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.526+0000 7fd6b7e00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009be0 tx=0x7fd6a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.527+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8039050 con 0x7fd69c0039b0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.527+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8045070 con 0x7fd69c0039b0 2024-04-12T19:51:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.527+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8003a50 con 0x7fd69c0039b0 2024-04-12T19:51:36.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.527+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b8143a80 con 0x7fd69c0039b0 2024-04-12T19:51:36.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.527+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b8144040 con 0x7fd69c0039b0 2024-04-12T19:51:36.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.528+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b80755c0 con 0x7fd69c0039b0 2024-04-12T19:51:36.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.529+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6a8003bf0 con 0x7fd69c0039b0 2024-04-12T19:51:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.531+0000 7fd6afe00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.531+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7fd6a80ad610 con 0x7fd69c0039b0 2024-04-12T19:51:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.531+0000 7fd6b7400640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.531+0000 7fd6b7400640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 0x7fd69806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006a90 tx=0x7fd6a4006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:36.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.536+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6a804e020 con 0x7fd69c0039b0 2024-04-12T19:51:36.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:36.784+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd6b80b5250 con 0x7fd69c0039b0 2024-04-12T19:51:37.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.304+0000 7fd6afe00640 1 -- 172.21.15.32:0/3362028831 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fd6b80b5a00 con 0x7fd69c0039b0 2024-04-12T19:51:37.308 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 4 and data pool 5 2024-04-12T19:51:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.317+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 msgr2=0x7fd69806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.317+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 0x7fd69806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006a90 tx=0x7fd6a4006a20 comp rx=0 tx=0).stop 2024-04-12T19:51:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.317+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 msgr2=0x7fd6b813d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.317+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009be0 tx=0x7fd6a8004730 comp rx=0 tx=0).stop 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 shutdown_connections 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd69806c8f0 0x7fd69806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b815ed30 0x7fd6b815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6b815e150 0x7fd6b815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 --2- 172.21.15.32:0/3362028831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69c0039b0 0x7fd6b813d120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 >> 172.21.15.32:0/3362028831 conn(0x7fd6b806b930 msgr2=0x7fd6b80af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:37.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.320+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 shutdown_connections 2024-04-12T19:51:37.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.323+0000 7fd6bdc00640 1 -- 172.21.15.32:0/3362028831 wait complete. 2024-04-12T19:51:37.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:37.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.536+0000 7f9e26a00640 1 Processor -- start 2024-04-12T19:51:37.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.537+0000 7f9e26a00640 1 -- start start 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.538+0000 7f9e26a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e20148130 0x7f9e2015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.538+0000 7f9e26a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e20072660 0x7f9e200af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.538+0000 7f9e26a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e20162940 con 0x7f9e20162380 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.538+0000 7f9e26a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9e20148510 con 0x7f9e20148130 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e26a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2015fd70 con 0x7f9e20072660 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e20148130 0x7f9e2015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e20148130 0x7f9e2015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52934/0 (socket says 172.21.15.32:52934) 2024-04-12T19:51:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 -- 172.21.15.32:0/2702687204 learned_addr learned my addr 172.21.15.32:0/2702687204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e26000640 1 --2- 172.21.15.32:0/2702687204 >> v2:172.21.15.169:3300/0 conn(0x7f9e20072660 0x7f9e200af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 -- 172.21.15.32:0/2702687204 >> v2:172.21.15.169:3300/0 conn(0x7f9e20072660 msgr2=0x7f9e200af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 --2- 172.21.15.32:0/2702687204 >> v2:172.21.15.169:3300/0 conn(0x7f9e20072660 0x7f9e200af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 -- 172.21.15.32:0/2702687204 >> v1:172.21.15.169:6789/0 conn(0x7f9e20162380 legacy=0x7f9e20071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:37.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.539+0000 7f9e25600640 1 -- 172.21.15.32:0/2702687204 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e20162c20 con 0x7f9e20148130 2024-04-12T19:51:37.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.541+0000 7f9e25600640 1 --2- 172.21.15.32:0/2702687204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20148130 0x7f9e2015f830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e2006ff10 tx=0x7f9e1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0fceade95732e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:37.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.541+0000 7f9e1fe00640 1 -- 172.21.15.32:0/2702687204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1402e070 con 0x7f9e20148130 2024-04-12T19:51:37.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.541+0000 7f9e1fe00640 1 -- 172.21.15.32:0/2702687204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14037070 con 0x7f9e20148130 2024-04-12T19:51:37.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.541+0000 7f9e1fe00640 1 -- 172.21.15.32:0/2702687204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14031a00 con 0x7f9e20148130 2024-04-12T19:51:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 -- 172.21.15.32:0/2702687204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20148130 msgr2=0x7f9e2015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 --2- 172.21.15.32:0/2702687204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20148130 0x7f9e2015f830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e2006ff10 tx=0x7f9e1402b880 comp rx=0 tx=0).stop 2024-04-12T19:51:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 -- 172.21.15.32:0/2702687204 shutdown_connections 2024-04-12T19:51:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 --2- 172.21.15.32:0/2702687204 >> v2:172.21.15.169:3300/0 conn(0x7f9e20072660 0x7f9e200af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 --2- 172.21.15.32:0/2702687204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20148130 0x7f9e2015f830 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 -- 172.21.15.32:0/2702687204 >> 172.21.15.32:0/2702687204 conn(0x7f9e2006b930 msgr2=0x7f9e2006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 -- 172.21.15.32:0/2702687204 shutdown_connections 2024-04-12T19:51:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.542+0000 7f9e26a00640 1 -- 172.21.15.32:0/2702687204 wait complete. 2024-04-12T19:51:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.544+0000 7f9e26a00640 1 Processor -- start 2024-04-12T19:51:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.544+0000 7f9e26a00640 1 -- start start 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e20148130 0x7f9e2014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e20162380 0x7f9e200c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200aeb90 con 0x7f9e20072660 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e201631e0 con 0x7f9e20148130 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e26a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e200aed10 con 0x7f9e20162380 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e25600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e25600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52944/0 (socket says 172.21.15.32:52944) 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.545+0000 7f9e25600640 1 -- 172.21.15.32:0/3296235283 learned_addr learned my addr 172.21.15.32:0/3296235283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 -- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e20162380 msgr2=0x7f9e200c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e20162380 0x7f9e200c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 -- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e20148130 msgr2=0x7f9e2014cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e20148130 0x7f9e2014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 -- 172.21.15.32:0/3296235283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e200c6ac0 con 0x7f9e20072660 2024-04-12T19:51:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e25600640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000c00 tx=0x7f9e140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14032880 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e14032a20 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14032e10 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.546+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e200b0820 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.547+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e200b0de0 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.548+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e200755c0 con 0x7f9e20072660 2024-04-12T19:51:37.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.549+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e1404c060 con 0x7f9e20072660 2024-04-12T19:51:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.550+0000 7f9e1d600640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 0x7f9e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.550+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9e140ad4f0 con 0x7f9e20072660 2024-04-12T19:51:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.550+0000 7f9e24c00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 0x7f9e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.551+0000 7f9e24c00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 0x7f9e0006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e20146e70 tx=0x7f9e0c006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.554+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e14040070 con 0x7f9e20072660 2024-04-12T19:51:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.761+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e2006f6d0 con 0x7f9e20072660 2024-04-12T19:51:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.763+0000 7f9e1d600640 1 -- 172.21.15.32:0/3296235283 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16610 (secure 0 0 0) 0x7f9e1407ba30 con 0x7f9e20072660 2024-04-12T19:51:37.764 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:37.764 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:36.786503+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 msgr2=0x7f9e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 0x7f9e0006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e20146e70 tx=0x7f9e0c006ba0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 msgr2=0x7f9e201f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000c00 tx=0x7f9e140046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 shutdown_connections 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9e0006c8f0 0x7f9e0006edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e20162380 0x7f9e200c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e20148130 0x7f9e2014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 --2- 172.21.15.32:0/3296235283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e20072660 0x7f9e201f7b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 >> 172.21.15.32:0/3296235283 conn(0x7f9e2006b930 msgr2=0x7f9e200b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.766+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 shutdown_connections 2024-04-12T19:51:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.767+0000 7f9e26a00640 1 -- 172.21.15.32:0/3296235283 wait complete. 2024-04-12T19:51:37.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.996+0000 7f840e400640 1 Processor -- start 2024-04-12T19:51:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.996+0000 7f840e400640 1 -- start start 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 0x7f840806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8408071f50 0x7f84080723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840e400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f840815dbb0 con 0x7f840815cbe0 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f840806f4f0 con 0x7f8408071f50 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84080728f0 con 0x7f84080cee00 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840da00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f840815cbe0 0x7f840815d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54408/0 (socket says 172.21.15.32:54408) 2024-04-12T19:51:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840da00640 1 -- 172.21.15.32:0/2161109552 learned_addr learned my addr 172.21.15.32:0/2161109552 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407e00640 1 --2- 172.21.15.32:0/2161109552 >> v2:172.21.15.169:3300/0 conn(0x7f8408071f50 0x7f84080723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059871236 0 0) 0x7f840815dbb0 con 0x7f840815cbe0 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840d000640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 0x7f840806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f0003520 con 0x7f840815cbe0 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961875034 0 0) 0x7f83f0003520 con 0x7f840815cbe0 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 msgr2=0x7f840806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 0x7f840806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 >> v2:172.21.15.169:3300/0 conn(0x7f8408071f50 msgr2=0x7f84080723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 --2- 172.21.15.32:0/2161109552 >> v2:172.21.15.169:3300/0 conn(0x7f8408071f50 0x7f84080723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840815d8d0 con 0x7f840815cbe0 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f84000019f0 con 0x7f840815cbe0 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 >> v1:172.21.15.32:6789/0 conn(0x7f840815cbe0 legacy=0x7f840815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 0x7f83f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0003fe0 0x7f83f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83f000ba70 con 0x7f83f0004d40 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83f000bbf0 con 0x7f83f0003fe0 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83f000bdc0 con 0x7f84080cee00 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.998+0000 7f840da00640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f8407e00640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0003fe0 0x7f83f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f840da00640 1 -- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 msgr2=0x7f83f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f840da00640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84080cee00 0x7f83f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f840da00640 1 -- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0003fe0 msgr2=0x7f83f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f840da00640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0003fe0 0x7f83f0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:37.999+0000 7f840da00640 1 -- 172.21.15.32:0/2161109552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f0003520 con 0x7f83f0004d40 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.000+0000 7f840da00640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f840000a860 tx=0x7f83f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e2df1ee3c4d427 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.000+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000a480 con 0x7f83f0004d40 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.000+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400034630 con 0x7f83f0004d40 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.000+0000 7f8407400640 1 -- 172.21.15.32:0/2161109552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400034a80 con 0x7f83f0004d40 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 -- 172.21.15.32:0/2161109552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f0004d40 msgr2=0x7f83f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 --2- 172.21.15.32:0/2161109552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f840000a860 tx=0x7f83f0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840da00640 1 -- 172.21.15.32:0/2161109552 reap_dead start 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 -- 172.21.15.32:0/2161109552 shutdown_connections 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 -- 172.21.15.32:0/2161109552 >> 172.21.15.32:0/2161109552 conn(0x7f840806b930 msgr2=0x7f840806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 -- 172.21.15.32:0/2161109552 shutdown_connections 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.001+0000 7f840e400640 1 -- 172.21.15.32:0/2161109552 wait complete. 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.004+0000 7f840e400640 1 Processor -- start 2024-04-12T19:51:38.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.004+0000 7f840e400640 1 -- start start 2024-04-12T19:51:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0004d40 0x7f84080c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8408148a10 0x7f8408148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f840806e3a0 con 0x7f840815e720 2024-04-12T19:51:38.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f840806e0c0 con 0x7f83f0004d40 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f840806e8c0 con 0x7f8408148a10 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52976/0 (socket says 172.21.15.32:52976) 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 -- 172.21.15.32:0/7825619 learned_addr learned my addr 172.21.15.32:0/7825619 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 -- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8408148a10 msgr2=0x7f8408148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8408148a10 0x7f8408148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 -- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0004d40 msgr2=0x7f84080c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840d000640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0004d40 0x7f84080c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0004d40 0x7f84080c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 -- 172.21.15.32:0/7825619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8408147c50 con 0x7f840815e720 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8407e00640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e7c0 tx=0x7f83f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4017070 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f400c6d0 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f400f040 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8408147f30 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.005+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081484f0 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.006+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84080ce0e0 con 0x7f840815e720 2024-04-12T19:51:38.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.008+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83f401a070 con 0x7f840815e720 2024-04-12T19:51:38.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.008+0000 7f8404c00640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 0x7f83e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.008+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f83f408b6c0 con 0x7f840815e720 2024-04-12T19:51:38.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.008+0000 7f840d000640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 0x7f83e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.009+0000 7f840d000640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 0x7f83e806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006a90 tx=0x7f83fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.012+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84080ce0e0 con 0x7f840815e720 2024-04-12T19:51:38.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.214+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84080755c0 con 0x7f840815e720 2024-04-12T19:51:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.215+0000 7f8404c00640 1 -- 172.21.15.32:0/7825619 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16610 (secure 0 0 0) 0x7f83f4059af0 con 0x7f840815e720 2024-04-12T19:51:38.215 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:38.215 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:36.786503+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 msgr2=0x7f83e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 0x7f83e806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006a90 tx=0x7f83fc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 msgr2=0x7f84080c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f83f400e7c0 tx=0x7f83f400ec80 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 shutdown_connections 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f83e806c8f0 0x7f83e806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8408148a10 0x7f8408148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f840815e720 0x7f84080c5b30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 --2- 172.21.15.32:0/7825619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83f0004d40 0x7f84080c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.218+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 >> 172.21.15.32:0/7825619 conn(0x7f840806b930 msgr2=0x7f84080b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.219+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 shutdown_connections 2024-04-12T19:51:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.219+0000 7f840e400640 1 -- 172.21.15.32:0/7825619 wait complete. 2024-04-12T19:51:38.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-12T19:51:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.457+0000 7f5841e00640 1 Processor -- start 2024-04-12T19:51:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.457+0000 7f5841e00640 1 -- start start 2024-04-12T19:51:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5841e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f583c0725f0 0x7f583c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5841e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5841e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f583c0cd2d0 con 0x7f583c1461c0 2024-04-12T19:51:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5841e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f583c0cf1d0 con 0x7f583c0725f0 2024-04-12T19:51:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5841e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f583c0cd080 con 0x7f583c0ce9e0 2024-04-12T19:51:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5833e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f583c0725f0 0x7f583c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5840a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f583c1461c0 0x7f583c071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54420/0 (socket says 172.21.15.32:54420) 2024-04-12T19:51:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.459+0000 7f5840a00640 1 -- 172.21.15.32:0/2300917543 learned_addr learned my addr 172.21.15.32:0/2300917543 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5841400640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5833e00640 1 -- 172.21.15.32:0/2300917543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 msgr2=0x7f583c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5833e00640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c06df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5833e00640 1 -- 172.21.15.32:0/2300917543 >> v1:172.21.15.32:6789/0 conn(0x7f583c1461c0 legacy=0x7f583c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5833e00640 1 -- 172.21.15.32:0/2300917543 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c0cd5b0 con 0x7f583c0725f0 2024-04-12T19:51:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.460+0000 7f5841400640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.461+0000 7f5833e00640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c0ce4a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f58240097a0 tx=0x7f582402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ff7dc25748a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.462+0000 7f583be00640 1 -- 172.21.15.32:0/2300917543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824035020 con 0x7f583c0725f0 2024-04-12T19:51:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.462+0000 7f583be00640 1 -- 172.21.15.32:0/2300917543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5824031810 con 0x7f583c0725f0 2024-04-12T19:51:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.462+0000 7f583be00640 1 -- 172.21.15.32:0/2300917543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824031be0 con 0x7f583c0725f0 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.462+0000 7f5841e00640 1 -- 172.21.15.32:0/2300917543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 msgr2=0x7f583c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.462+0000 7f5841e00640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c0ce4a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f58240097a0 tx=0x7f582402b8b0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 -- 172.21.15.32:0/2300917543 shutdown_connections 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 --2- 172.21.15.32:0/2300917543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c0ce4a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 -- 172.21.15.32:0/2300917543 >> 172.21.15.32:0/2300917543 conn(0x7f583c06b930 msgr2=0x7f583c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 -- 172.21.15.32:0/2300917543 shutdown_connections 2024-04-12T19:51:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.463+0000 7f5841e00640 1 -- 172.21.15.32:0/2300917543 wait complete. 2024-04-12T19:51:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.465+0000 7f5841e00640 1 Processor -- start 2024-04-12T19:51:38.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.465+0000 7f5841e00640 1 -- start start 2024-04-12T19:51:38.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c1efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 0x7f583c1f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f583c0cee70 con 0x7f583c0725f0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f583c0cdb70 con 0x7f583c1461c0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f583c06e460 con 0x7f583c0ce9e0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53002/0 (socket says 172.21.15.32:53002) 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 -- 172.21.15.32:0/286650350 learned_addr learned my addr 172.21.15.32:0/286650350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5833e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c1efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5841400640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 0x7f583c1f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 -- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 msgr2=0x7f583c1efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c1efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 -- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 msgr2=0x7f583c1f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.466+0000 7f5840a00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 0x7f583c1f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5840a00640 1 -- 172.21.15.32:0/286650350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c132ce0 con 0x7f583c0725f0 2024-04-12T19:51:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5841400640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 0x7f583c1f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5840a00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f582c00ca70 tx=0x7f582c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c017070 con 0x7f583c0725f0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c00ab70 con 0x7f583c0725f0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c012400 con 0x7f583c0725f0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c132fc0 con 0x7f583c0725f0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.467+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c157810 con 0x7f583c0725f0 2024-04-12T19:51:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.468+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583c0755c0 con 0x7f583c0725f0 2024-04-12T19:51:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.470+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f582c01a070 con 0x7f583c0725f0 2024-04-12T19:51:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.470+0000 7f5839600640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 0x7f581806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.471+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f582c08b8b0 con 0x7f583c0725f0 2024-04-12T19:51:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.471+0000 7f5833e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 0x7f581806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.472+0000 7f5833e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 0x7f581806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5824036000 tx=0x7f5824008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.476+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f582c059cd0 con 0x7f583c0725f0 2024-04-12T19:51:38.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:38.716+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f583c072e40 con 0x7f583c0725f0 2024-04-12T19:51:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.320+0000 7f5839600640 1 -- 172.21.15.32:0/286650350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f582c0595d0 con 0x7f583c0725f0 2024-04-12T19:51:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 msgr2=0x7f581806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 0x7f581806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5824036000 tx=0x7f5824008030 comp rx=0 tx=0).stop 2024-04-12T19:51:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 msgr2=0x7f583c1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f582c00ca70 tx=0x7f582c00cf30 comp rx=0 tx=0).stop 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 shutdown_connections 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f581806c8f0 0x7f581806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f583c1461c0 0x7f583c1f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f583c0ce9e0 0x7f583c1efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 --2- 172.21.15.32:0/286650350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0725f0 0x7f583c1572d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 >> 172.21.15.32:0/286650350 conn(0x7f583c06b930 msgr2=0x7f583c1453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 shutdown_connections 2024-04-12T19:51:39.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.324+0000 7f5841e00640 1 -- 172.21.15.32:0/286650350 wait complete. 2024-04-12T19:51:39.346 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-12T19:51:39.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-12T19:51:39.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.576+0000 7f53f1600640 1 Processor -- start 2024-04-12T19:51:39.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.576+0000 7f53f1600640 1 -- start start 2024-04-12T19:51:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.577+0000 7f53f1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 0x7f53ec071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.577+0000 7f53f1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53ec0cebb0 0x7f53ec06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.577+0000 7f53f1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53ec0cd2d0 con 0x7f53ec072420 2024-04-12T19:51:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.577+0000 7f53f1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f53ec0cf1d0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.577+0000 7f53f1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53ec072880 con 0x7f53ec1461c0 2024-04-12T19:51:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.578+0000 7f53f0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53ec0cebb0 0x7f53ec06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.578+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 0x7f53ec071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.578+0000 7f53f0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53ec0cebb0 0x7f53ec06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:42540/0 (socket says 172.21.15.32:42540) 2024-04-12T19:51:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.578+0000 7f53ebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 0x7f53ec071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:36994/0 (socket says 172.21.15.32:36994) 2024-04-12T19:51:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.578+0000 7f53f0c00640 1 -- 172.21.15.32:0/1978131613 learned_addr learned my addr 172.21.15.32:0/1978131613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.579+0000 7f53f0c00640 1 -- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 msgr2=0x7f53ec071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.579+0000 7f53f0c00640 1 --2- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 0x7f53ec071ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.579+0000 7f53f0c00640 1 -- 172.21.15.32:0/1978131613 >> v1:172.21.15.169:6789/0 conn(0x7f53ec072420 legacy=0x7f53ec0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.579+0000 7f53f0c00640 1 -- 172.21.15.32:0/1978131613 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec0cd5b0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.580+0000 7f53f0c00640 1 --2- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec0cebb0 0x7f53ec06df20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009e50 tx=0x7f53e402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753042aa12120397 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.581+0000 7f53eaa00640 1 -- 172.21.15.32:0/1978131613 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e402cb00 con 0x7f53ec0cebb0 2024-04-12T19:51:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.581+0000 7f53eaa00640 1 -- 172.21.15.32:0/1978131613 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e402cca0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.581+0000 7f53eaa00640 1 -- 172.21.15.32:0/1978131613 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4034610 con 0x7f53ec0cebb0 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.581+0000 7f53f1600640 1 -- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec0cebb0 msgr2=0x7f53ec06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.581+0000 7f53f1600640 1 --2- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec0cebb0 0x7f53ec06df20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009e50 tx=0x7f53e402bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 -- 172.21.15.32:0/1978131613 shutdown_connections 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 --2- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec0cebb0 0x7f53ec06df20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 --2- 172.21.15.32:0/1978131613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec1461c0 0x7f53ec071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 -- 172.21.15.32:0/1978131613 >> 172.21.15.32:0/1978131613 conn(0x7f53ec06b930 msgr2=0x7f53ec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 -- 172.21.15.32:0/1978131613 shutdown_connections 2024-04-12T19:51:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.582+0000 7f53f1600640 1 -- 172.21.15.32:0/1978131613 wait complete. 2024-04-12T19:51:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.583+0000 7f53f1600640 1 Processor -- start 2024-04-12T19:51:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.583+0000 7f53f1600640 1 -- start start 2024-04-12T19:51:39.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec072420 0x7f53ec143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 0x7f53ec141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec0cdb70 con 0x7f53ec0cebb0 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec0cd080 con 0x7f53ec1461c0 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.584+0000 7f53f1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53ec06e460 con 0x7f53ec072420 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53016/0 (socket says 172.21.15.32:53016) 2024-04-12T19:51:39.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 -- 172.21.15.32:0/2687390771 learned_addr learned my addr 172.21.15.32:0/2687390771 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53ebe00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec072420 0x7f53ec143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53f0c00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 0x7f53ec141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 -- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec072420 msgr2=0x7f53ec143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec072420 0x7f53ec143750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 -- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 msgr2=0x7f53ec141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 0x7f53ec141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53eb400640 1 -- 172.21.15.32:0/2687390771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1423f0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.585+0000 7f53f0c00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 0x7f53ec141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53eb400640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c430 tx=0x7f53d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d80052d0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8005490 con 0x7f53ec0cebb0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8007260 con 0x7f53ec0cebb0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec142690 con 0x7f53ec0cebb0 2024-04-12T19:51:39.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.586+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec143170 con 0x7f53ec0cebb0 2024-04-12T19:51:39.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.587+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec0755c0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.588+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53d8007400 con 0x7f53ec0cebb0 2024-04-12T19:51:39.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.589+0000 7f53e3e00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:39.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.590+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f53d808b0b0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.590+0000 7f53ebe00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:39.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.590+0000 7f53ebe00640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 0x7f53cc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a30 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:39.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.594+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53d80594e0 con 0x7f53ec0cebb0 2024-04-12T19:51:39.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:39.842+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f53ec0b20f0 con 0x7f53ec0cebb0 2024-04-12T19:51:40.102 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-12T19:51:40.103+0000 7f1aac000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-12T19:51:40.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:51:40.121 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:51:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.334+0000 7f53e3e00640 1 -- 172.21.15.32:0/2687390771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f53d8058de0 con 0x7f53ec0cebb0 2024-04-12T19:51:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 msgr2=0x7f53cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 0x7f53cc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009a30 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2024-04-12T19:51:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 msgr2=0x7f53ec140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c430 tx=0x7f53d800c8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 shutdown_connections 2024-04-12T19:51:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.337+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53ec1461c0 0x7f53ec141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec0cebb0 0x7f53ec140e50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 --2- 172.21.15.32:0/2687390771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53ec072420 0x7f53ec143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 >> 172.21.15.32:0/2687390771 conn(0x7f53ec06b930 msgr2=0x7f53ec15bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 shutdown_connections 2024-04-12T19:51:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.338+0000 7f53f1600640 1 -- 172.21.15.32:0/2687390771 wait complete. 2024-04-12T19:51:40.363 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-12T19:51:40.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.589+0000 7f347be00640 1 Processor -- start 2024-04-12T19:51:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.590+0000 7f347be00640 1 -- start start 2024-04-12T19:51:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.591+0000 7f347be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f347c147690 0x7f347c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.591+0000 7f347be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f347c070930 0x7f347c0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.591+0000 7f347be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0b6180 con 0x7f347c06b930 2024-04-12T19:51:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.591+0000 7f347be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0b6740 con 0x7f347c070930 2024-04-12T19:51:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.591+0000 7f347be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0b6460 con 0x7f347c147690 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f3473e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f347c070930 0x7f347c0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f347b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f347c06b930 0x7f347c06bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54434/0 (socket says 172.21.15.32:54434) 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f347b400640 1 -- 172.21.15.32:0/958669977 learned_addr learned my addr 172.21.15.32:0/958669977 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f347aa00640 1 --2- 172.21.15.32:0/958669977 >> v2:172.21.15.169:3300/0 conn(0x7f347c147690 0x7f347c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f3473e00640 1 -- 172.21.15.32:0/958669977 >> v2:172.21.15.169:3300/0 conn(0x7f347c147690 msgr2=0x7f347c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f3473e00640 1 --2- 172.21.15.32:0/958669977 >> v2:172.21.15.169:3300/0 conn(0x7f347c147690 0x7f347c0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f3473e00640 1 -- 172.21.15.32:0/958669977 >> v1:172.21.15.32:6789/0 conn(0x7f347c06b930 legacy=0x7f347c06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f347a000640 1 -- 172.21.15.32:0/958669977 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974991630 0 0) 0x7f347c0b6180 con 0x7f347c06b930 2024-04-12T19:51:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.592+0000 7f3473e00640 1 -- 172.21.15.32:0/958669977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c06f1b0 con 0x7f347c070930 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.594+0000 7f3473e00640 1 --2- 172.21.15.32:0/958669977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c0ae3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f347c072610 tx=0x7f346402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac543858e9321ca server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.594+0000 7f347a000640 1 -- 172.21.15.32:0/958669977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464035020 con 0x7f347c070930 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.594+0000 7f347a000640 1 -- 172.21.15.32:0/958669977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3464031830 con 0x7f347c070930 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.594+0000 7f347a000640 1 -- 172.21.15.32:0/958669977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464031c00 con 0x7f347c070930 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 -- 172.21.15.32:0/958669977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 msgr2=0x7f347c0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 --2- 172.21.15.32:0/958669977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c0ae3b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f347c072610 tx=0x7f346402b8d0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 -- 172.21.15.32:0/958669977 shutdown_connections 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 --2- 172.21.15.32:0/958669977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c0ae3b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 --2- 172.21.15.32:0/958669977 >> v2:172.21.15.169:3300/0 conn(0x7f347c147690 0x7f347c0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 -- 172.21.15.32:0/958669977 >> 172.21.15.32:0/958669977 conn(0x7f347c09aa40 msgr2=0x7f347c0b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 -- 172.21.15.32:0/958669977 shutdown_connections 2024-04-12T19:51:40.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.595+0000 7f347be00640 1 -- 172.21.15.32:0/958669977 wait complete. 2024-04-12T19:51:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.597+0000 7f347be00640 1 Processor -- start 2024-04-12T19:51:40.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.597+0000 7f347be00640 1 -- start start 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f347c06b930 0x7f347c09dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f347c147690 0x7f347c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0aea70 con 0x7f347c070930 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0ae8f0 con 0x7f347c06b930 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f347c070dc0 con 0x7f347c147690 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f3473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f3473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53048/0 (socket says 172.21.15.32:53048) 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f3473e00640 1 -- 172.21.15.32:0/3658261092 learned_addr learned my addr 172.21.15.32:0/3658261092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.598+0000 7f347aa00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f347c06b930 0x7f347c09dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f347b400640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f347c147690 0x7f347c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 -- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f347c147690 msgr2=0x7f347c09b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f347c147690 0x7f347c09b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 -- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f347c06b930 msgr2=0x7f347c09dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f347c06b930 0x7f347c09dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 -- 172.21.15.32:0/3658261092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c09e110 con 0x7f347c070930 2024-04-12T19:51:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3473e00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f346402bee0 tx=0x7f34640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464041070 con 0x7f347c070930 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346403f070 con 0x7f347c070930 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.599+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464035040 con 0x7f347c070930 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.600+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c09e3f0 con 0x7f347c070930 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.600+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c09e9b0 con 0x7f347c070930 2024-04-12T19:51:40.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.601+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c0755c0 con 0x7f347c070930 2024-04-12T19:51:40.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.602+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3464038020 con 0x7f347c070930 2024-04-12T19:51:40.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.603+0000 7f3472a00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 0x7f345806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:40.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.603+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f34640ad2f0 con 0x7f347c070930 2024-04-12T19:51:40.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.603+0000 7f347aa00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 0x7f345806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:40.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.604+0000 7f347aa00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 0x7f345806ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f346c006fd0 tx=0x7f346c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.607+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f346407b680 con 0x7f347c070930 2024-04-12T19:51:40.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.851+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347c1448b0 con 0x7f347c070930 2024-04-12T19:51:40.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.853+0000 7f3472a00640 1 -- 172.21.15.32:0/3658261092 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f346407af80 con 0x7f347c070930 2024-04-12T19:51:40.852 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:40.853 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:51:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:51:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 msgr2=0x7f345806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 0x7f345806ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f346c006fd0 tx=0x7f346c008040 comp rx=0 tx=0).stop 2024-04-12T19:51:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 msgr2=0x7f347c13feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f346402bee0 tx=0x7f34640046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 shutdown_connections 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f345806c920 0x7f345806ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f347c147690 0x7f347c09b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347c070930 0x7f347c13feb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 --2- 172.21.15.32:0/3658261092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f347c06b930 0x7f347c09dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 >> 172.21.15.32:0/3658261092 conn(0x7f347c09aa40 msgr2=0x7f347c143840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 shutdown_connections 2024-04-12T19:51:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:40.856+0000 7f347be00640 1 -- 172.21.15.32:0/3658261092 wait complete. 2024-04-12T19:51:40.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.075+0000 7f886a200640 1 Processor -- start 2024-04-12T19:51:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.075+0000 7f886a200640 1 -- start start 2024-04-12T19:51:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.077+0000 7f886a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.077+0000 7f886a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 0x7f88640d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.077+0000 7f886a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88640d0090 con 0x7f88640cf110 2024-04-12T19:51:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.077+0000 7f886a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8864163440 con 0x7f88640d1c90 2024-04-12T19:51:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.077+0000 7f886a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88640b7b30 con 0x7f8864162020 2024-04-12T19:51:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8869800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 0x7f88640d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:37024/0 (socket says 172.21.15.32:37024) 2024-04-12T19:51:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 -- 172.21.15.32:0/2214255208 learned_addr learned my addr 172.21.15.32:0/2214255208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 -- 172.21.15.32:0/2214255208 >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 msgr2=0x7f88640d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 --2- 172.21.15.32:0/2214255208 >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 0x7f88640d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 -- 172.21.15.32:0/2214255208 >> v1:172.21.15.169:6789/0 conn(0x7f88640cf110 legacy=0x7f88640d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8868e00640 1 -- 172.21.15.32:0/2214255208 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88640cfad0 con 0x7f8864162020 2024-04-12T19:51:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.078+0000 7f8869800640 1 --2- 172.21.15.32:0/2214255208 >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 0x7f88640d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.080+0000 7f8868e00640 1 --2- 172.21.15.32:0/2214255208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8858009e50 tx=0x7f885802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f16e7d895431f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.080+0000 7f8863400640 1 -- 172.21.15.32:0/2214255208 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885802cb00 con 0x7f8864162020 2024-04-12T19:51:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.080+0000 7f8863400640 1 -- 172.21.15.32:0/2214255208 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885802cca0 con 0x7f8864162020 2024-04-12T19:51:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.080+0000 7f8863400640 1 -- 172.21.15.32:0/2214255208 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858034610 con 0x7f8864162020 2024-04-12T19:51:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 -- 172.21.15.32:0/2214255208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 msgr2=0x7f8864162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 --2- 172.21.15.32:0/2214255208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8858009e50 tx=0x7f885802bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 -- 172.21.15.32:0/2214255208 shutdown_connections 2024-04-12T19:51:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 --2- 172.21.15.32:0/2214255208 >> v2:172.21.15.169:3300/0 conn(0x7f88640d1c90 0x7f88640d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 --2- 172.21.15.32:0/2214255208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f8864162f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 -- 172.21.15.32:0/2214255208 >> 172.21.15.32:0/2214255208 conn(0x7f886406cf00 msgr2=0x7f886406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 -- 172.21.15.32:0/2214255208 shutdown_connections 2024-04-12T19:51:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.081+0000 7f886a200640 1 -- 172.21.15.32:0/2214255208 wait complete. 2024-04-12T19:51:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.083+0000 7f886a200640 1 Processor -- start 2024-04-12T19:51:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.083+0000 7f886a200640 1 -- start start 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88640cf110 0x7f88640c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f88641570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f88640d2b90 con 0x7f88640d1c90 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88640cf880 con 0x7f88640cf110 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f886a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88640d2e70 con 0x7f8864162020 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f8863e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f8863e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53050/0 (socket says 172.21.15.32:53050) 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.084+0000 7f8863e00640 1 -- 172.21.15.32:0/658884704 learned_addr learned my addr 172.21.15.32:0/658884704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8868e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88640cf110 0x7f88640c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8869800640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f88641570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 -- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 msgr2=0x7f88641570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f88641570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 -- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88640cf110 msgr2=0x7f88640c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88640cf110 0x7f88640c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 -- 172.21.15.32:0/658884704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88641557c0 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.085+0000 7f8863e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f885000c3c0 tx=0x7f885000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.086+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88500052d0 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.086+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8850005490 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.086+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8864155aa0 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.086+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850007260 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.086+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8864156060 con 0x7f88640d1c90 2024-04-12T19:51:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.087+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88640755c0 con 0x7f88640d1c90 2024-04-12T19:51:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.088+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8850007400 con 0x7f88640d1c90 2024-04-12T19:51:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.089+0000 7f8860c00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 0x7f884406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.089+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f885008aeb0 con 0x7f88640d1c90 2024-04-12T19:51:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.089+0000 7f8868e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 0x7f884406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.090+0000 7f8868e00640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 0x7f884406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8858035000 tx=0x7f8858030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.093+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88640d0090 con 0x7f88640d1c90 2024-04-12T19:51:41.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.293+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88640b7650 con 0x7f88640d1c90 2024-04-12T19:51:41.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.296+0000 7f8860c00640 1 -- 172.21.15.32:0/658884704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f885005f020 con 0x7f88640d1c90 2024-04-12T19:51:41.295 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:41.296 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:38.305869+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.298+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 msgr2=0x7f884406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.298+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 0x7f884406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8858035000 tx=0x7f8858030000 comp rx=0 tx=0).stop 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.298+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 msgr2=0x7f8864154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.298+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f885000c3c0 tx=0x7f885000c880 comp rx=0 tx=0).stop 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 shutdown_connections 2024-04-12T19:51:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f884406c8f0 0x7f884406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8864162020 0x7f88641570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88640d1c90 0x7f8864154de0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 --2- 172.21.15.32:0/658884704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88640cf110 0x7f88640c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 >> 172.21.15.32:0/658884704 conn(0x7f886406cf00 msgr2=0x7f88640cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 shutdown_connections 2024-04-12T19:51:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.299+0000 7f886a200640 1 -- 172.21.15.32:0/658884704 wait complete. 2024-04-12T19:51:41.317 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-12T19:51:41.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:51:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.503+0000 7f3bb1e00640 1 Processor -- start 2024-04-12T19:51:41.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.503+0000 7f3bb1e00640 1 -- start start 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.504+0000 7f3bb1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3bac163060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.504+0000 7f3bb1e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac1635a0 con 0x7f3bac0b3c70 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.504+0000 7f3bb1e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac15ef90 con 0x7f3bac0d1cc0 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.504+0000 7f3bb1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf090 con 0x7f3bac15ebb0 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.505+0000 7f3babe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3bac0b3c70 0x7f3bac0d15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54446/0 (socket says 172.21.15.32:54446) 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.505+0000 7f3babe00640 1 -- 172.21.15.32:0/2662280415 learned_addr learned my addr 172.21.15.32:0/2662280415 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.505+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3bac163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.505+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80721866 0 0) 0x7f3bac1635a0 con 0x7f3bac0b3c70 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.505+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b94003520 con 0x7f3bac0b3c70 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2028656061 0 0) 0x7f3b94003520 con 0x7f3bac0b3c70 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 msgr2=0x7f3bac163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3bac163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 >> v1:172.21.15.169:6789/0 conn(0x7f3bac0d1cc0 legacy=0x7f3bac0d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0cf370 con 0x7f3bac0b3c70 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f3ba0001670 con 0x7f3bac0b3c70 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 >> v1:172.21.15.32:6789/0 conn(0x7f3bac0b3c70 legacy=0x7f3bac0d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf090 con 0x7f3b940039e0 2024-04-12T19:51:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9400ba70 con 0x7f3b94004d40 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9400bc70 con 0x7f3bac15ebb0 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.506+0000 7f3babe00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb0a00640 1 -- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 msgr2=0x7f3b94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac15ebb0 0x7f3b94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb1400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb0a00640 1 -- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 msgr2=0x7f3b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb0a00640 1 -- 172.21.15.32:0/2662280415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b94003520 con 0x7f3b940039e0 2024-04-12T19:51:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.507+0000 7f3bb1400640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.508+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009a90 tx=0x7f3b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee20b371914ab9b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.508+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c039070 con 0x7f3b940039e0 2024-04-12T19:51:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.508+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9c02c680 con 0x7f3b940039e0 2024-04-12T19:51:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.508+0000 7f3bab400640 1 -- 172.21.15.32:0/2662280415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c009690 con 0x7f3b940039e0 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 -- 172.21.15.32:0/2662280415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b940039e0 msgr2=0x7f3b94003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/2662280415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b940039e0 0x7f3b94003e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009a90 tx=0x7f3b94005830 comp rx=0 tx=0).stop 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1400640 1 -- 172.21.15.32:0/2662280415 reap_dead start 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 -- 172.21.15.32:0/2662280415 shutdown_connections 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 -- 172.21.15.32:0/2662280415 >> 172.21.15.32:0/2662280415 conn(0x7f3bac06d040 msgr2=0x7f3bac06d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 -- 172.21.15.32:0/2662280415 shutdown_connections 2024-04-12T19:51:41.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.509+0000 7f3bb1e00640 1 -- 172.21.15.32:0/2662280415 wait complete. 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.511+0000 7f3bb1e00640 1 Processor -- start 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.511+0000 7f3bb1e00640 1 -- start start 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b940039e0 0x7f3bac152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 0x7f3bac0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cfc10 con 0x7f3bac0b4290 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf930 con 0x7f3b940039e0 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0cfef0 con 0x7f3bac0cd200 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3babe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3babe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53068/0 (socket says 172.21.15.32:53068) 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3babe00640 1 -- 172.21.15.32:0/510852013 learned_addr learned my addr 172.21.15.32:0/510852013 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.512+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b940039e0 0x7f3bac152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3bb1400640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 0x7f3bac0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 -- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 msgr2=0x7f3bac0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 0x7f3bac0cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 -- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b940039e0 msgr2=0x7f3bac152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b940039e0 0x7f3bac152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 -- 172.21.15.32:0/510852013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0b4cf0 con 0x7f3bac0b4290 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3bb1400640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 0x7f3bac0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:41.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3babe00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e790 tx=0x7f3ba000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000eea0 con 0x7f3bac0b4290 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0018070 con 0x7f3bac0b4290 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000ca90 con 0x7f3bac0b4290 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.513+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac0b4fd0 con 0x7f3bac0b4290 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.514+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac0b5590 con 0x7f3bac0b4290 2024-04-12T19:51:41.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.515+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac0755c0 con 0x7f3bac0b4290 2024-04-12T19:51:41.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.516+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ba000cc30 con 0x7f3bac0b4290 2024-04-12T19:51:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.517+0000 7f3ba8c00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 0x7f3b9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.517+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3ba008b8a0 con 0x7f3bac0b4290 2024-04-12T19:51:41.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.517+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 0x7f3b9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.518+0000 7f3bb0a00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 0x7f3b9006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c036000 tx=0x7f3b9c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.521+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ba0059d60 con 0x7f3bac0b4290 2024-04-12T19:51:41.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.717+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bac0d2540 con 0x7f3b9006c8f0 2024-04-12T19:51:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.737+0000 7f3ba8c00640 1 -- 172.21.15.32:0/510852013 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247145 (secure 0 0 0) 0x7f3bac0d2540 con 0x7f3b9006c8f0 2024-04-12T19:51:41.739 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:51:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.741+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 msgr2=0x7f3b9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.741+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 0x7f3b9006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c036000 tx=0x7f3b9c030000 comp rx=0 tx=0).stop 2024-04-12T19:51:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.741+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 msgr2=0x7f3bac0b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.741+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e790 tx=0x7f3ba000ec50 comp rx=0 tx=0).stop 2024-04-12T19:51:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 shutdown_connections 2024-04-12T19:51:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f3b9006c8f0 0x7f3b9006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0cd200 0x7f3bac0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bac0b4290 0x7f3bac0b46f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 --2- 172.21.15.32:0/510852013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b940039e0 0x7f3bac152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 >> 172.21.15.32:0/510852013 conn(0x7f3bac06d040 msgr2=0x7f3bac0ba150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 shutdown_connections 2024-04-12T19:51:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:41.742+0000 7f3bb1e00640 1 -- 172.21.15.32:0/510852013 wait complete. 2024-04-12T19:51:41.758 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-12T19:51:41.649894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236792,"kb_used_data":3976,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737928,"statfs":{"total":773094113280,"available":772851638272,"internally_reserved":0,"allocated":4071424,"data_stored":2269634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001544"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302279+0000","last_change":"2024-04-12T19:51:36.302279+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302279+0000","last_undegraded":"2024-04-12T19:51:36.302279+0000","last_fullsized":"2024-04-12T19:51:36.302279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314104+0000","last_change":"2024-04-12T19:51:36.314104+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314104+0000","last_undegraded":"2024-04-12T19:51:36.314104+0000","last_fullsized":"2024-04-12T19:51:36.314104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.310776+0000","last_change":"2024-04-12T19:51:36.310776+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.310776+0000","last_undegraded":"2024-04-12T19:51:36.310776+0000","last_fullsized":"2024-04-12T19:51:36.310776+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.307884+0000","last_change":"2024-04-12T19:51:36.307884+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.307884+0000","last_undegraded":"2024-04-12T19:51:36.307884+0000","last_fullsized":"2024-04-12T19:51:36.307884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.319707+0000","last_change":"2024-04-12T19:51:36.319707+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.319707+0000","last_undegraded":"2024-04-12T19:51:36.319707+0000","last_fullsized":"2024-04-12T19:51:36.319707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327988+0000","last_change":"2024-04-12T19:51:37.335976+0000","last_active":"2024-04-12T19:51:38.327988+0000","last_peered":"2024-04-12T19:51:38.327988+0000","last_clean":"2024-04-12T19:51:38.327988+0000","last_became_active":"2024-04-12T19:51:37.335672+0000","last_became_peered":"2024-04-12T19:51:37.335672+0000","last_unstale":"2024-04-12T19:51:38.327988+0000","last_undegraded":"2024-04-12T19:51:38.327988+0000","last_fullsized":"2024-04-12T19:51:38.327988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:11:45.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303201+0000","last_change":"2024-04-12T19:51:36.303201+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303201+0000","last_undegraded":"2024-04-12T19:51:36.303201+0000","last_fullsized":"2024-04-12T19:51:36.303201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306771+0000","last_change":"2024-04-12T19:51:36.306771+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306771+0000","last_undegraded":"2024-04-12T19:51:36.306771+0000","last_fullsized":"2024-04-12T19:51:36.306771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309809+0000","last_change":"2024-04-12T19:51:36.309809+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309809+0000","last_undegraded":"2024-04-12T19:51:36.309809+0000","last_fullsized":"2024-04-12T19:51:36.309809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305225+0000","last_change":"2024-04-12T19:51:36.305225+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305225+0000","last_undegraded":"2024-04-12T19:51:36.305225+0000","last_fullsized":"2024-04-12T19:51:36.305225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304052+0000","last_change":"2024-04-12T19:51:36.304052+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304052+0000","last_undegraded":"2024-04-12T19:51:36.304052+0000","last_fullsized":"2024-04-12T19:51:36.304052+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309056+0000","last_change":"2024-04-12T19:51:36.309056+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309056+0000","last_undegraded":"2024-04-12T19:51:36.309056+0000","last_fullsized":"2024-04-12T19:51:36.309056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314265+0000","last_change":"2024-04-12T19:51:36.314265+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314265+0000","last_undegraded":"2024-04-12T19:51:36.314265+0000","last_fullsized":"2024-04-12T19:51:36.314265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.300858+0000","last_change":"2024-04-12T19:51:36.300858+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.300858+0000","last_undegraded":"2024-04-12T19:51:36.300858+0000","last_fullsized":"2024-04-12T19:51:36.300858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297764+0000","last_change":"2024-04-12T19:51:36.297764+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297764+0000","last_undegraded":"2024-04-12T19:51:36.297764+0000","last_fullsized":"2024-04-12T19:51:36.297764+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297452+0000","last_change":"2024-04-12T19:51:36.297452+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297452+0000","last_undegraded":"2024-04-12T19:51:36.297452+0000","last_fullsized":"2024-04-12T19:51:36.297452+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303096+0000","last_change":"2024-04-12T19:51:36.303096+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303096+0000","last_undegraded":"2024-04-12T19:51:36.303096+0000","last_fullsized":"2024-04-12T19:51:36.303096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327881+0000","last_change":"2024-04-12T19:51:37.326355+0000","last_active":"2024-04-12T19:51:38.327881+0000","last_peered":"2024-04-12T19:51:38.327881+0000","last_clean":"2024-04-12T19:51:38.327881+0000","last_became_active":"2024-04-12T19:51:37.326026+0000","last_became_peered":"2024-04-12T19:51:37.326026+0000","last_unstale":"2024-04-12T19:51:38.327881+0000","last_undegraded":"2024-04-12T19:51:38.327881+0000","last_fullsized":"2024-04-12T19:51:38.327881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:15:16.329928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328156+0000","last_change":"2024-04-12T19:51:37.328370+0000","last_active":"2024-04-12T19:51:38.328156+0000","last_peered":"2024-04-12T19:51:38.328156+0000","last_clean":"2024-04-12T19:51:38.328156+0000","last_became_active":"2024-04-12T19:51:37.328016+0000","last_became_peered":"2024-04-12T19:51:37.328016+0000","last_unstale":"2024-04-12T19:51:38.328156+0000","last_undegraded":"2024-04-12T19:51:38.328156+0000","last_fullsized":"2024-04-12T19:51:38.328156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:59:33.416853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.322194+0000","last_change":"2024-04-12T19:51:36.322194+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.322194+0000","last_undegraded":"2024-04-12T19:51:36.322194+0000","last_fullsized":"2024-04-12T19:51:36.322194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.298505+0000","last_change":"2024-04-12T19:51:36.298505+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.298505+0000","last_undegraded":"2024-04-12T19:51:36.298505+0000","last_fullsized":"2024-04-12T19:51:36.298505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305444+0000","last_change":"2024-04-12T19:51:36.305444+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305444+0000","last_undegraded":"2024-04-12T19:51:36.305444+0000","last_fullsized":"2024-04-12T19:51:36.305444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316919+0000","last_change":"2024-04-12T19:51:36.316919+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316919+0000","last_undegraded":"2024-04-12T19:51:36.316919+0000","last_fullsized":"2024-04-12T19:51:36.316919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303515+0000","last_change":"2024-04-12T19:51:36.303515+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303515+0000","last_undegraded":"2024-04-12T19:51:36.303515+0000","last_fullsized":"2024-04-12T19:51:36.303515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305132+0000","last_change":"2024-04-12T19:51:36.305132+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305132+0000","last_undegraded":"2024-04-12T19:51:36.305132+0000","last_fullsized":"2024-04-12T19:51:36.305132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304008+0000","last_change":"2024-04-12T19:51:36.304008+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304008+0000","last_undegraded":"2024-04-12T19:51:36.304008+0000","last_fullsized":"2024-04-12T19:51:36.304008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317199+0000","last_change":"2024-04-12T19:51:36.317199+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317199+0000","last_undegraded":"2024-04-12T19:51:36.317199+0000","last_fullsized":"2024-04-12T19:51:36.317199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306291+0000","last_change":"2024-04-12T19:51:36.306291+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306291+0000","last_undegraded":"2024-04-12T19:51:36.306291+0000","last_fullsized":"2024-04-12T19:51:36.306291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297996+0000","last_change":"2024-04-12T19:51:36.297996+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297996+0000","last_undegraded":"2024-04-12T19:51:36.297996+0000","last_fullsized":"2024-04-12T19:51:36.297996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297423+0000","last_change":"2024-04-12T19:51:36.297423+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297423+0000","last_undegraded":"2024-04-12T19:51:36.297423+0000","last_fullsized":"2024-04-12T19:51:36.297423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301892+0000","last_change":"2024-04-12T19:51:36.301892+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301892+0000","last_undegraded":"2024-04-12T19:51:36.301892+0000","last_fullsized":"2024-04-12T19:51:36.301892+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302517+0000","last_change":"2024-04-12T19:51:36.302517+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302517+0000","last_undegraded":"2024-04-12T19:51:36.302517+0000","last_fullsized":"2024-04-12T19:51:36.302517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.318539+0000","last_change":"2024-04-12T19:51:36.318539+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.318539+0000","last_undegraded":"2024-04-12T19:51:36.318539+0000","last_fullsized":"2024-04-12T19:51:36.318539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.324592+0000","last_change":"2024-04-12T19:51:36.323028+0000","last_active":"2024-04-12T19:51:36.324592+0000","last_peered":"2024-04-12T19:51:36.324592+0000","last_clean":"2024-04-12T19:51:36.324592+0000","last_became_active":"2024-04-12T19:51:36.322694+0000","last_became_peered":"2024-04-12T19:51:36.322694+0000","last_unstale":"2024-04-12T19:51:36.324592+0000","last_undegraded":"2024-04-12T19:51:36.324592+0000","last_fullsized":"2024-04-12T19:51:36.324592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:09.348012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301568+0000","last_change":"2024-04-12T19:51:36.301568+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301568+0000","last_undegraded":"2024-04-12T19:51:36.301568+0000","last_fullsized":"2024-04-12T19:51:36.301568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330065+0000","last_change":"2024-04-12T19:51:36.325298+0000","last_active":"2024-04-12T19:51:36.330065+0000","last_peered":"2024-04-12T19:51:36.330065+0000","last_clean":"2024-04-12T19:51:36.330065+0000","last_became_active":"2024-04-12T19:51:36.324222+0000","last_became_peered":"2024-04-12T19:51:36.324222+0000","last_unstale":"2024-04-12T19:51:36.330065+0000","last_undegraded":"2024-04-12T19:51:36.330065+0000","last_fullsized":"2024-04-12T19:51:36.330065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301690+0000","last_change":"2024-04-12T19:51:36.301690+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301690+0000","last_undegraded":"2024-04-12T19:51:36.301690+0000","last_fullsized":"2024-04-12T19:51:36.301690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328355+0000","last_change":"2024-04-12T19:51:36.331707+0000","last_active":"2024-04-12T19:51:38.328355+0000","last_peered":"2024-04-12T19:51:38.328355+0000","last_clean":"2024-04-12T19:51:38.328355+0000","last_became_active":"2024-04-12T19:51:36.331198+0000","last_became_peered":"2024-04-12T19:51:36.331198+0000","last_unstale":"2024-04-12T19:51:38.328355+0000","last_undegraded":"2024-04-12T19:51:38.328355+0000","last_fullsized":"2024-04-12T19:51:38.328355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:18.552643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305586+0000","last_change":"2024-04-12T19:51:36.305586+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305586+0000","last_undegraded":"2024-04-12T19:51:36.305586+0000","last_fullsized":"2024-04-12T19:51:36.305586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.336724+0000","last_change":"2024-04-12T19:51:36.330056+0000","last_active":"2024-04-12T19:51:36.336724+0000","last_peered":"2024-04-12T19:51:36.336724+0000","last_clean":"2024-04-12T19:51:36.336724+0000","last_became_active":"2024-04-12T19:51:36.329657+0000","last_became_peered":"2024-04-12T19:51:36.329657+0000","last_unstale":"2024-04-12T19:51:36.336724+0000","last_undegraded":"2024-04-12T19:51:36.336724+0000","last_fullsized":"2024-04-12T19:51:36.336724+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308524+0000","last_change":"2024-04-12T19:51:36.308524+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308524+0000","last_undegraded":"2024-04-12T19:51:36.308524+0000","last_fullsized":"2024-04-12T19:51:36.308524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328738+0000","last_change":"2024-04-12T19:51:36.327409+0000","last_active":"2024-04-12T19:51:36.328738+0000","last_peered":"2024-04-12T19:51:36.328738+0000","last_clean":"2024-04-12T19:51:36.328738+0000","last_became_active":"2024-04-12T19:51:36.327107+0000","last_became_peered":"2024-04-12T19:51:36.327107+0000","last_unstale":"2024-04-12T19:51:36.328738+0000","last_undegraded":"2024-04-12T19:51:36.328738+0000","last_fullsized":"2024-04-12T19:51:36.328738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306806+0000","last_change":"2024-04-12T19:51:36.306806+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306806+0000","last_undegraded":"2024-04-12T19:51:36.306806+0000","last_fullsized":"2024-04-12T19:51:36.306806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327470+0000","last_change":"2024-04-12T19:51:36.324064+0000","last_active":"2024-04-12T19:51:36.327470+0000","last_peered":"2024-04-12T19:51:36.327470+0000","last_clean":"2024-04-12T19:51:36.327470+0000","last_became_active":"2024-04-12T19:51:36.323334+0000","last_became_peered":"2024-04-12T19:51:36.323334+0000","last_unstale":"2024-04-12T19:51:36.327470+0000","last_undegraded":"2024-04-12T19:51:36.327470+0000","last_fullsized":"2024-04-12T19:51:36.327470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:48:02.416908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.313566+0000","last_change":"2024-04-12T19:51:36.313566+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.313566+0000","last_undegraded":"2024-04-12T19:51:36.313566+0000","last_fullsized":"2024-04-12T19:51:36.313566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326879+0000","last_change":"2024-04-12T19:51:36.326080+0000","last_active":"2024-04-12T19:51:38.326879+0000","last_peered":"2024-04-12T19:51:38.326879+0000","last_clean":"2024-04-12T19:51:38.326879+0000","last_became_active":"2024-04-12T19:51:36.325817+0000","last_became_peered":"2024-04-12T19:51:36.325817+0000","last_unstale":"2024-04-12T19:51:38.326879+0000","last_undegraded":"2024-04-12T19:51:38.326879+0000","last_fullsized":"2024-04-12T19:51:38.326879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:03.515939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302461+0000","last_change":"2024-04-12T19:51:36.302461+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302461+0000","last_undegraded":"2024-04-12T19:51:36.302461+0000","last_fullsized":"2024-04-12T19:51:36.302461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322424+0000","last_change":"2024-04-12T19:51:36.320855+0000","last_active":"2024-04-12T19:51:36.322424+0000","last_peered":"2024-04-12T19:51:36.322424+0000","last_clean":"2024-04-12T19:51:36.322424+0000","last_became_active":"2024-04-12T19:51:36.320566+0000","last_became_peered":"2024-04-12T19:51:36.320566+0000","last_unstale":"2024-04-12T19:51:36.322424+0000","last_undegraded":"2024-04-12T19:51:36.322424+0000","last_fullsized":"2024-04-12T19:51:36.322424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:43:56.356046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303073+0000","last_change":"2024-04-12T19:51:36.303073+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303073+0000","last_undegraded":"2024-04-12T19:51:36.303073+0000","last_fullsized":"2024-04-12T19:51:36.303073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328522+0000","last_change":"2024-04-12T19:51:36.323202+0000","last_active":"2024-04-12T19:51:36.328522+0000","last_peered":"2024-04-12T19:51:36.328522+0000","last_clean":"2024-04-12T19:51:36.328522+0000","last_became_active":"2024-04-12T19:51:36.322883+0000","last_became_peered":"2024-04-12T19:51:36.322883+0000","last_unstale":"2024-04-12T19:51:36.328522+0000","last_undegraded":"2024-04-12T19:51:36.328522+0000","last_fullsized":"2024-04-12T19:51:36.328522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302674+0000","last_change":"2024-04-12T19:51:36.302674+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302674+0000","last_undegraded":"2024-04-12T19:51:36.302674+0000","last_fullsized":"2024-04-12T19:51:36.302674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332954+0000","last_change":"2024-04-12T19:51:36.327948+0000","last_active":"2024-04-12T19:51:36.332954+0000","last_peered":"2024-04-12T19:51:36.332954+0000","last_clean":"2024-04-12T19:51:36.332954+0000","last_became_active":"2024-04-12T19:51:36.327654+0000","last_became_peered":"2024-04-12T19:51:36.327654+0000","last_unstale":"2024-04-12T19:51:36.332954+0000","last_undegraded":"2024-04-12T19:51:36.332954+0000","last_fullsized":"2024-04-12T19:51:36.332954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:44:36.049413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306597+0000","last_change":"2024-04-12T19:51:36.306597+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306597+0000","last_undegraded":"2024-04-12T19:51:36.306597+0000","last_fullsized":"2024-04-12T19:51:36.306597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.309507+0000","last_change":"2024-04-12T19:51:36.304808+0000","last_active":"2024-04-12T19:51:36.309507+0000","last_peered":"2024-04-12T19:51:36.309507+0000","last_clean":"2024-04-12T19:51:36.309507+0000","last_became_active":"2024-04-12T19:51:36.304470+0000","last_became_peered":"2024-04-12T19:51:36.304470+0000","last_unstale":"2024-04-12T19:51:36.309507+0000","last_undegraded":"2024-04-12T19:51:36.309507+0000","last_fullsized":"2024-04-12T19:51:36.309507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:12.187706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301837+0000","last_change":"2024-04-12T19:51:36.301837+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301837+0000","last_undegraded":"2024-04-12T19:51:36.301837+0000","last_fullsized":"2024-04-12T19:51:36.301837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337157+0000","last_change":"2024-04-12T19:51:36.331502+0000","last_active":"2024-04-12T19:51:36.337157+0000","last_peered":"2024-04-12T19:51:36.337157+0000","last_clean":"2024-04-12T19:51:36.337157+0000","last_became_active":"2024-04-12T19:51:36.329297+0000","last_became_peered":"2024-04-12T19:51:36.329297+0000","last_unstale":"2024-04-12T19:51:36.337157+0000","last_undegraded":"2024-04-12T19:51:36.337157+0000","last_fullsized":"2024-04-12T19:51:36.337157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.320204+0000","last_change":"2024-04-12T19:51:36.320204+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.320204+0000","last_undegraded":"2024-04-12T19:51:36.320204+0000","last_fullsized":"2024-04-12T19:51:36.320204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318782+0000","last_change":"2024-04-12T19:51:36.316118+0000","last_active":"2024-04-12T19:51:36.318782+0000","last_peered":"2024-04-12T19:51:36.318782+0000","last_clean":"2024-04-12T19:51:36.318782+0000","last_became_active":"2024-04-12T19:51:36.315852+0000","last_became_peered":"2024-04-12T19:51:36.315852+0000","last_unstale":"2024-04-12T19:51:36.318782+0000","last_undegraded":"2024-04-12T19:51:36.318782+0000","last_fullsized":"2024-04-12T19:51:36.318782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:57:20.899617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305931+0000","last_change":"2024-04-12T19:51:36.305931+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305931+0000","last_undegraded":"2024-04-12T19:51:36.305931+0000","last_fullsized":"2024-04-12T19:51:36.305931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.325983+0000","last_change":"2024-04-12T19:51:36.320973+0000","last_active":"2024-04-12T19:51:36.325983+0000","last_peered":"2024-04-12T19:51:36.325983+0000","last_clean":"2024-04-12T19:51:36.325983+0000","last_became_active":"2024-04-12T19:51:36.320688+0000","last_became_peered":"2024-04-12T19:51:36.320688+0000","last_unstale":"2024-04-12T19:51:36.325983+0000","last_undegraded":"2024-04-12T19:51:36.325983+0000","last_fullsized":"2024-04-12T19:51:36.325983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302481+0000","last_change":"2024-04-12T19:51:36.302481+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302481+0000","last_undegraded":"2024-04-12T19:51:36.302481+0000","last_fullsized":"2024-04-12T19:51:36.302481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.323574+0000","last_change":"2024-04-12T19:51:36.313661+0000","last_active":"2024-04-12T19:51:36.323574+0000","last_peered":"2024-04-12T19:51:36.323574+0000","last_clean":"2024-04-12T19:51:36.323574+0000","last_became_active":"2024-04-12T19:51:36.313249+0000","last_became_peered":"2024-04-12T19:51:36.313249+0000","last_unstale":"2024-04-12T19:51:36.323574+0000","last_undegraded":"2024-04-12T19:51:36.323574+0000","last_fullsized":"2024-04-12T19:51:36.323574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301377+0000","last_change":"2024-04-12T19:51:36.301377+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301377+0000","last_undegraded":"2024-04-12T19:51:36.301377+0000","last_fullsized":"2024-04-12T19:51:36.301377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326281+0000","last_change":"2024-04-12T19:51:36.324874+0000","last_active":"2024-04-12T19:51:38.326281+0000","last_peered":"2024-04-12T19:51:38.326281+0000","last_clean":"2024-04-12T19:51:38.326281+0000","last_became_active":"2024-04-12T19:51:36.324013+0000","last_became_peered":"2024-04-12T19:51:36.324013+0000","last_unstale":"2024-04-12T19:51:38.326281+0000","last_undegraded":"2024-04-12T19:51:38.326281+0000","last_fullsized":"2024-04-12T19:51:38.326281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:10:23.725348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297465+0000","last_change":"2024-04-12T19:51:36.297465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297465+0000","last_undegraded":"2024-04-12T19:51:36.297465+0000","last_fullsized":"2024-04-12T19:51:36.297465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328103+0000","last_change":"2024-04-12T19:51:36.332183+0000","last_active":"2024-04-12T19:51:38.328103+0000","last_peered":"2024-04-12T19:51:38.328103+0000","last_clean":"2024-04-12T19:51:38.328103+0000","last_became_active":"2024-04-12T19:51:36.331021+0000","last_became_peered":"2024-04-12T19:51:36.331021+0000","last_unstale":"2024-04-12T19:51:38.328103+0000","last_undegraded":"2024-04-12T19:51:38.328103+0000","last_fullsized":"2024-04-12T19:51:38.328103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:29:38.531795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309373+0000","last_change":"2024-04-12T19:51:36.309373+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309373+0000","last_undegraded":"2024-04-12T19:51:36.309373+0000","last_fullsized":"2024-04-12T19:51:36.309373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.312675+0000","last_change":"2024-04-12T19:51:36.311352+0000","last_active":"2024-04-12T19:51:36.312675+0000","last_peered":"2024-04-12T19:51:36.312675+0000","last_clean":"2024-04-12T19:51:36.312675+0000","last_became_active":"2024-04-12T19:51:36.310541+0000","last_became_peered":"2024-04-12T19:51:36.310541+0000","last_unstale":"2024-04-12T19:51:36.312675+0000","last_undegraded":"2024-04-12T19:51:36.312675+0000","last_fullsized":"2024-04-12T19:51:36.312675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.312465+0000","last_change":"2024-04-12T19:51:36.312465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.312465+0000","last_undegraded":"2024-04-12T19:51:36.312465+0000","last_fullsized":"2024-04-12T19:51:36.312465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.321500+0000","last_change":"2024-04-12T19:51:36.320077+0000","last_active":"2024-04-12T19:51:36.321500+0000","last_peered":"2024-04-12T19:51:36.321500+0000","last_clean":"2024-04-12T19:51:36.321500+0000","last_became_active":"2024-04-12T19:51:36.319763+0000","last_became_peered":"2024-04-12T19:51:36.319763+0000","last_unstale":"2024-04-12T19:51:36.321500+0000","last_undegraded":"2024-04-12T19:51:36.321500+0000","last_fullsized":"2024-04-12T19:51:36.321500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:38.560305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328222+0000","last_change":"2024-04-12T19:51:37.318524+0000","last_active":"2024-04-12T19:51:38.328222+0000","last_peered":"2024-04-12T19:51:38.328222+0000","last_clean":"2024-04-12T19:51:38.328222+0000","last_became_active":"2024-04-12T19:51:37.318222+0000","last_became_peered":"2024-04-12T19:51:37.318222+0000","last_unstale":"2024-04-12T19:51:38.328222+0000","last_undegraded":"2024-04-12T19:51:38.328222+0000","last_fullsized":"2024-04-12T19:51:38.328222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:10:43.194581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318050+0000","last_change":"2024-04-12T19:51:36.315549+0000","last_active":"2024-04-12T19:51:36.318050+0000","last_peered":"2024-04-12T19:51:36.318050+0000","last_clean":"2024-04-12T19:51:36.318050+0000","last_became_active":"2024-04-12T19:51:36.314815+0000","last_became_peered":"2024-04-12T19:51:36.314815+0000","last_unstale":"2024-04-12T19:51:36.318050+0000","last_undegraded":"2024-04-12T19:51:36.318050+0000","last_fullsized":"2024-04-12T19:51:36.318050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317974+0000","last_change":"2024-04-12T19:51:36.317974+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317974+0000","last_undegraded":"2024-04-12T19:51:36.317974+0000","last_fullsized":"2024-04-12T19:51:36.317974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332929+0000","last_change":"2024-04-12T19:51:36.327171+0000","last_active":"2024-04-12T19:51:36.332929+0000","last_peered":"2024-04-12T19:51:36.332929+0000","last_clean":"2024-04-12T19:51:36.332929+0000","last_became_active":"2024-04-12T19:51:36.326818+0000","last_became_peered":"2024-04-12T19:51:36.326818+0000","last_unstale":"2024-04-12T19:51:36.332929+0000","last_undegraded":"2024-04-12T19:51:36.332929+0000","last_fullsized":"2024-04-12T19:51:36.332929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316050+0000","last_change":"2024-04-12T19:51:36.316050+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316050+0000","last_undegraded":"2024-04-12T19:51:36.316050+0000","last_fullsized":"2024-04-12T19:51:36.316050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318733+0000","last_change":"2024-04-12T19:51:36.315615+0000","last_active":"2024-04-12T19:51:36.318733+0000","last_peered":"2024-04-12T19:51:36.318733+0000","last_clean":"2024-04-12T19:51:36.318733+0000","last_became_active":"2024-04-12T19:51:36.315325+0000","last_became_peered":"2024-04-12T19:51:36.315325+0000","last_unstale":"2024-04-12T19:51:36.318733+0000","last_undegraded":"2024-04-12T19:51:36.318733+0000","last_fullsized":"2024-04-12T19:51:36.318733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:49.798145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.315697+0000","last_change":"2024-04-12T19:51:36.315697+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.315697+0000","last_undegraded":"2024-04-12T19:51:36.315697+0000","last_fullsized":"2024-04-12T19:51:36.315697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329354+0000","last_change":"2024-04-12T19:51:36.326548+0000","last_active":"2024-04-12T19:51:36.329354+0000","last_peered":"2024-04-12T19:51:36.329354+0000","last_clean":"2024-04-12T19:51:36.329354+0000","last_became_active":"2024-04-12T19:51:36.326046+0000","last_became_peered":"2024-04-12T19:51:36.326046+0000","last_unstale":"2024-04-12T19:51:36.329354+0000","last_undegraded":"2024-04-12T19:51:36.329354+0000","last_fullsized":"2024-04-12T19:51:36.329354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308731+0000","last_change":"2024-04-12T19:51:36.308731+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308731+0000","last_undegraded":"2024-04-12T19:51:36.308731+0000","last_fullsized":"2024-04-12T19:51:36.308731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322237+0000","last_change":"2024-04-12T19:51:36.321134+0000","last_active":"2024-04-12T19:51:36.322237+0000","last_peered":"2024-04-12T19:51:36.322237+0000","last_clean":"2024-04-12T19:51:36.322237+0000","last_became_active":"2024-04-12T19:51:36.320811+0000","last_became_peered":"2024-04-12T19:51:36.320811+0000","last_unstale":"2024-04-12T19:51:36.322237+0000","last_undegraded":"2024-04-12T19:51:36.322237+0000","last_fullsized":"2024-04-12T19:51:36.322237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:16.117219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327966+0000","last_change":"2024-04-12T19:51:37.331351+0000","last_active":"2024-04-12T19:51:38.327966+0000","last_peered":"2024-04-12T19:51:38.327966+0000","last_clean":"2024-04-12T19:51:38.327966+0000","last_became_active":"2024-04-12T19:51:37.330895+0000","last_became_peered":"2024-04-12T19:51:37.330895+0000","last_unstale":"2024-04-12T19:51:38.327966+0000","last_undegraded":"2024-04-12T19:51:38.327966+0000","last_fullsized":"2024-04-12T19:51:38.327966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:44:51.414187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330180+0000","last_change":"2024-04-12T19:51:36.327741+0000","last_active":"2024-04-12T19:51:36.330180+0000","last_peered":"2024-04-12T19:51:36.330180+0000","last_clean":"2024-04-12T19:51:36.330180+0000","last_became_active":"2024-04-12T19:51:36.327226+0000","last_became_peered":"2024-04-12T19:51:36.327226+0000","last_unstale":"2024-04-12T19:51:36.330180+0000","last_undegraded":"2024-04-12T19:51:36.330180+0000","last_fullsized":"2024-04-12T19:51:36.330180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296972+0000","last_change":"2024-04-12T19:51:36.296972+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296972+0000","last_undegraded":"2024-04-12T19:51:36.296972+0000","last_fullsized":"2024-04-12T19:51:36.296972+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329105+0000","last_change":"2024-04-12T19:51:36.324717+0000","last_active":"2024-04-12T19:51:36.329105+0000","last_peered":"2024-04-12T19:51:36.329105+0000","last_clean":"2024-04-12T19:51:36.329105+0000","last_became_active":"2024-04-12T19:51:36.324437+0000","last_became_peered":"2024-04-12T19:51:36.324437+0000","last_unstale":"2024-04-12T19:51:36.329105+0000","last_undegraded":"2024-04-12T19:51:36.329105+0000","last_fullsized":"2024-04-12T19:51:36.329105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328579+0000","last_change":"2024-04-12T19:51:37.316433+0000","last_active":"2024-04-12T19:51:38.328579+0000","last_peered":"2024-04-12T19:51:38.328579+0000","last_clean":"2024-04-12T19:51:38.328579+0000","last_became_active":"2024-04-12T19:51:37.316089+0000","last_became_peered":"2024-04-12T19:51:37.316089+0000","last_unstale":"2024-04-12T19:51:38.328579+0000","last_undegraded":"2024-04-12T19:51:38.328579+0000","last_fullsized":"2024-04-12T19:51:38.328579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:26:38.058351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337134+0000","last_change":"2024-04-12T19:51:36.331177+0000","last_active":"2024-04-12T19:51:36.337134+0000","last_peered":"2024-04-12T19:51:36.337134+0000","last_clean":"2024-04-12T19:51:36.337134+0000","last_became_active":"2024-04-12T19:51:36.329431+0000","last_became_peered":"2024-04-12T19:51:36.329431+0000","last_unstale":"2024-04-12T19:51:36.337134+0000","last_undegraded":"2024-04-12T19:51:36.337134+0000","last_fullsized":"2024-04-12T19:51:36.337134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296734+0000","last_change":"2024-04-12T19:51:36.296734+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296734+0000","last_undegraded":"2024-04-12T19:51:36.296734+0000","last_fullsized":"2024-04-12T19:51:36.296734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327735+0000","last_change":"2024-04-12T19:51:36.321573+0000","last_active":"2024-04-12T19:51:36.327735+0000","last_peered":"2024-04-12T19:51:36.327735+0000","last_clean":"2024-04-12T19:51:36.327735+0000","last_became_active":"2024-04-12T19:51:36.321273+0000","last_became_peered":"2024-04-12T19:51:36.321273+0000","last_unstale":"2024-04-12T19:51:36.327735+0000","last_undegraded":"2024-04-12T19:51:36.327735+0000","last_fullsized":"2024-04-12T19:51:36.327735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304472+0000","last_change":"2024-04-12T19:51:36.304472+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304472+0000","last_undegraded":"2024-04-12T19:51:36.304472+0000","last_fullsized":"2024-04-12T19:51:36.304472+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327327+0000","last_change":"2024-04-12T19:51:36.323335+0000","last_active":"2024-04-12T19:51:36.327327+0000","last_peered":"2024-04-12T19:51:36.327327+0000","last_clean":"2024-04-12T19:51:36.327327+0000","last_became_active":"2024-04-12T19:51:36.323010+0000","last_became_peered":"2024-04-12T19:51:36.323010+0000","last_unstale":"2024-04-12T19:51:36.327327+0000","last_undegraded":"2024-04-12T19:51:36.327327+0000","last_fullsized":"2024-04-12T19:51:36.327327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:40:21.315726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302102+0000","last_change":"2024-04-12T19:51:36.302102+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302102+0000","last_undegraded":"2024-04-12T19:51:36.302102+0000","last_fullsized":"2024-04-12T19:51:36.302102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318537+0000","last_change":"2024-04-12T19:51:36.315742+0000","last_active":"2024-04-12T19:51:36.318537+0000","last_peered":"2024-04-12T19:51:36.318537+0000","last_clean":"2024-04-12T19:51:36.318537+0000","last_became_active":"2024-04-12T19:51:36.315459+0000","last_became_peered":"2024-04-12T19:51:36.315459+0000","last_unstale":"2024-04-12T19:51:36.318537+0000","last_undegraded":"2024-04-12T19:51:36.318537+0000","last_fullsized":"2024-04-12T19:51:36.318537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:40.771292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297673+0000","last_change":"2024-04-12T19:51:36.297673+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297673+0000","last_undegraded":"2024-04-12T19:51:36.297673+0000","last_fullsized":"2024-04-12T19:51:36.297673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329868+0000","last_change":"2024-04-12T19:51:36.324603+0000","last_active":"2024-04-12T19:51:36.329868+0000","last_peered":"2024-04-12T19:51:36.329868+0000","last_clean":"2024-04-12T19:51:36.329868+0000","last_became_active":"2024-04-12T19:51:36.322196+0000","last_became_peered":"2024-04-12T19:51:36.322196+0000","last_unstale":"2024-04-12T19:51:36.329868+0000","last_undegraded":"2024-04-12T19:51:36.329868+0000","last_fullsized":"2024-04-12T19:51:36.329868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296831+0000","last_change":"2024-04-12T19:51:36.296831+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296831+0000","last_undegraded":"2024-04-12T19:51:36.296831+0000","last_fullsized":"2024-04-12T19:51:36.296831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322184+0000","last_change":"2024-04-12T19:51:36.320886+0000","last_active":"2024-04-12T19:51:36.322184+0000","last_peered":"2024-04-12T19:51:36.322184+0000","last_clean":"2024-04-12T19:51:36.322184+0000","last_became_active":"2024-04-12T19:51:36.320612+0000","last_became_peered":"2024-04-12T19:51:36.320612+0000","last_unstale":"2024-04-12T19:51:36.322184+0000","last_undegraded":"2024-04-12T19:51:36.322184+0000","last_fullsized":"2024-04-12T19:51:36.322184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:51.567339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.314747+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:51:36.314747+0000","last_peered":"2024-04-12T19:51:36.314747+0000","last_clean":"2024-04-12T19:51:36.314747+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:51:36.314747+0000","last_undegraded":"2024-04-12T19:51:36.314747+0000","last_fullsized":"2024-04-12T19:51:36.314747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":728765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":138397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":722490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:51:41.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:51:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.013+0000 7f0855200640 1 Processor -- start 2024-04-12T19:51:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.014+0000 7f0855200640 1 -- start start 2024-04-12T19:51:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f0855200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 0x7f0850162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f0855200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08500d0090 con 0x7f0850162020 2024-04-12T19:51:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f0855200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08500cfad0 con 0x7f08500cf110 2024-04-12T19:51:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f0855200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0850163440 con 0x7f08500d1c90 2024-04-12T19:51:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f084f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0850162020 0x7f08500d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54450/0 (socket says 172.21.15.32:54450) 2024-04-12T19:51:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.015+0000 7f084f400640 1 -- 172.21.15.32:0/2355229181 learned_addr learned my addr 172.21.15.32:0/2355229181 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129747928 0 0) 0x7f0840001110 con 0x7f0850162020 2024-04-12T19:51:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f0847e00640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 0x7f0850162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0834003520 con 0x7f0850162020 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4224768203 0 0) 0x7f0834003520 con 0x7f0850162020 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 msgr2=0x7f0850162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 0x7f0850162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 >> v1:172.21.15.169:6789/0 conn(0x7f08500cf110 legacy=0x7f08500d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08500cfdb0 con 0x7f0850162020 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f0840001840 con 0x7f0850162020 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 >> v1:172.21.15.32:6789/0 conn(0x7f0850162020 legacy=0x7f08500d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08340039e0 0x7f0834003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 0x7f0834004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0834004d40 0x7f08340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0850163440 con 0x7f08340039e0 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f083400bb50 con 0x7f0834004d40 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.016+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f083400bd50 con 0x7f08500d1c90 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08340039e0 0x7f0834003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 -- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 msgr2=0x7f0834004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08500d1c90 0x7f0834004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 -- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0834004d40 msgr2=0x7f08340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0834004d40 0x7f08340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.017+0000 7f084f400640 1 -- 172.21.15.32:0/2355229181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0834003520 con 0x7f08340039e0 2024-04-12T19:51:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.018+0000 7f084f400640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08340039e0 0x7f0834003de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f084000c320 tx=0x7f0834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ff77f3203dd4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.018+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084002cc40 con 0x7f08340039e0 2024-04-12T19:51:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.018+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0840030070 con 0x7f08340039e0 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.018+0000 7f084ea00640 1 -- 172.21.15.32:0/2355229181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08400058d0 con 0x7f08340039e0 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 -- 172.21.15.32:0/2355229181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08340039e0 msgr2=0x7f0834003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 --2- 172.21.15.32:0/2355229181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08340039e0 0x7f0834003de0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f084000c320 tx=0x7f0834005830 comp rx=0 tx=0).stop 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f084fe00640 1 -- 172.21.15.32:0/2355229181 reap_dead start 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 -- 172.21.15.32:0/2355229181 shutdown_connections 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 -- 172.21.15.32:0/2355229181 >> 172.21.15.32:0/2355229181 conn(0x7f085006cf00 msgr2=0x7f085006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 -- 172.21.15.32:0/2355229181 shutdown_connections 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.019+0000 7f0855200640 1 -- 172.21.15.32:0/2355229181 wait complete. 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.021+0000 7f0855200640 1 Processor -- start 2024-04-12T19:51:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.021+0000 7f0855200640 1 -- start start 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08340039e0 0x7f08501588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0850157cd0 0x7f08500c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08500cf880 con 0x7f0850158130 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08500b7b30 con 0x7f0850157cd0 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.022+0000 7f0855200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08500d2e70 con 0x7f08340039e0 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53082/0 (socket says 172.21.15.32:53082) 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 -- 172.21.15.32:0/4001646751 learned_addr learned my addr 172.21.15.32:0/4001646751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f0847e00640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0850157cd0 0x7f08500c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084f400640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08340039e0 0x7f08501588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 -- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08340039e0 msgr2=0x7f08501588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08340039e0 0x7f08501588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 -- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0850157cd0 msgr2=0x7f08500c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0850157cd0 0x7f08500c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 -- 172.21.15.32:0/4001646751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0850156c30 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.023+0000 7f084fe00640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0834004780 tx=0x7f084800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.024+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848017070 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.024+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0848005d40 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.024+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084800a850 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.024+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0850156f10 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.024+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501574d0 con 0x7f0850158130 2024-04-12T19:51:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.025+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08500755c0 con 0x7f0850158130 2024-04-12T19:51:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.026+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084801a070 con 0x7f0850158130 2024-04-12T19:51:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.027+0000 7f0847400640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.027+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f084808aea0 con 0x7f0850158130 2024-04-12T19:51:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.027+0000 7f084f400640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.028+0000 7f084f400640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 0x7f082c06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0840034000 tx=0x7f0840039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.031+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08480591a0 con 0x7f0850158130 2024-04-12T19:51:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.225+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08500b7650 con 0x7f082c06c920 2024-04-12T19:51:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.232+0000 7f0847400640 1 -- 172.21.15.32:0/4001646751 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247145 (secure 0 0 0) 0x7f08500b7650 con 0x7f082c06c920 2024-04-12T19:51:42.235 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:51:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 msgr2=0x7f082c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 0x7f082c06ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0840034000 tx=0x7f0840039040 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 msgr2=0x7f08500c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0834004780 tx=0x7f084800cb90 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 shutdown_connections 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0850158130 0x7f08500c6f50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0850157cd0 0x7f08500c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 --2- 172.21.15.32:0/4001646751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08340039e0 0x7f08501588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 >> 172.21.15.32:0/4001646751 conn(0x7f085006cf00 msgr2=0x7f08500cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 shutdown_connections 2024-04-12T19:51:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.238+0000 7f0855200640 1 -- 172.21.15.32:0/4001646751 wait complete. 2024-04-12T19:51:42.256 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-12T19:51:41.649894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236792,"kb_used_data":3976,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737928,"statfs":{"total":773094113280,"available":772851638272,"internally_reserved":0,"allocated":4071424,"data_stored":2269634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001544"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302279+0000","last_change":"2024-04-12T19:51:36.302279+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302279+0000","last_undegraded":"2024-04-12T19:51:36.302279+0000","last_fullsized":"2024-04-12T19:51:36.302279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314104+0000","last_change":"2024-04-12T19:51:36.314104+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314104+0000","last_undegraded":"2024-04-12T19:51:36.314104+0000","last_fullsized":"2024-04-12T19:51:36.314104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.310776+0000","last_change":"2024-04-12T19:51:36.310776+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.310776+0000","last_undegraded":"2024-04-12T19:51:36.310776+0000","last_fullsized":"2024-04-12T19:51:36.310776+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.307884+0000","last_change":"2024-04-12T19:51:36.307884+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.307884+0000","last_undegraded":"2024-04-12T19:51:36.307884+0000","last_fullsized":"2024-04-12T19:51:36.307884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.319707+0000","last_change":"2024-04-12T19:51:36.319707+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.319707+0000","last_undegraded":"2024-04-12T19:51:36.319707+0000","last_fullsized":"2024-04-12T19:51:36.319707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327988+0000","last_change":"2024-04-12T19:51:37.335976+0000","last_active":"2024-04-12T19:51:38.327988+0000","last_peered":"2024-04-12T19:51:38.327988+0000","last_clean":"2024-04-12T19:51:38.327988+0000","last_became_active":"2024-04-12T19:51:37.335672+0000","last_became_peered":"2024-04-12T19:51:37.335672+0000","last_unstale":"2024-04-12T19:51:38.327988+0000","last_undegraded":"2024-04-12T19:51:38.327988+0000","last_fullsized":"2024-04-12T19:51:38.327988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:11:45.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303201+0000","last_change":"2024-04-12T19:51:36.303201+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303201+0000","last_undegraded":"2024-04-12T19:51:36.303201+0000","last_fullsized":"2024-04-12T19:51:36.303201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306771+0000","last_change":"2024-04-12T19:51:36.306771+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306771+0000","last_undegraded":"2024-04-12T19:51:36.306771+0000","last_fullsized":"2024-04-12T19:51:36.306771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309809+0000","last_change":"2024-04-12T19:51:36.309809+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309809+0000","last_undegraded":"2024-04-12T19:51:36.309809+0000","last_fullsized":"2024-04-12T19:51:36.309809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305225+0000","last_change":"2024-04-12T19:51:36.305225+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305225+0000","last_undegraded":"2024-04-12T19:51:36.305225+0000","last_fullsized":"2024-04-12T19:51:36.305225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304052+0000","last_change":"2024-04-12T19:51:36.304052+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304052+0000","last_undegraded":"2024-04-12T19:51:36.304052+0000","last_fullsized":"2024-04-12T19:51:36.304052+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309056+0000","last_change":"2024-04-12T19:51:36.309056+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309056+0000","last_undegraded":"2024-04-12T19:51:36.309056+0000","last_fullsized":"2024-04-12T19:51:36.309056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314265+0000","last_change":"2024-04-12T19:51:36.314265+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314265+0000","last_undegraded":"2024-04-12T19:51:36.314265+0000","last_fullsized":"2024-04-12T19:51:36.314265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.300858+0000","last_change":"2024-04-12T19:51:36.300858+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.300858+0000","last_undegraded":"2024-04-12T19:51:36.300858+0000","last_fullsized":"2024-04-12T19:51:36.300858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297764+0000","last_change":"2024-04-12T19:51:36.297764+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297764+0000","last_undegraded":"2024-04-12T19:51:36.297764+0000","last_fullsized":"2024-04-12T19:51:36.297764+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297452+0000","last_change":"2024-04-12T19:51:36.297452+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297452+0000","last_undegraded":"2024-04-12T19:51:36.297452+0000","last_fullsized":"2024-04-12T19:51:36.297452+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303096+0000","last_change":"2024-04-12T19:51:36.303096+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303096+0000","last_undegraded":"2024-04-12T19:51:36.303096+0000","last_fullsized":"2024-04-12T19:51:36.303096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327881+0000","last_change":"2024-04-12T19:51:37.326355+0000","last_active":"2024-04-12T19:51:38.327881+0000","last_peered":"2024-04-12T19:51:38.327881+0000","last_clean":"2024-04-12T19:51:38.327881+0000","last_became_active":"2024-04-12T19:51:37.326026+0000","last_became_peered":"2024-04-12T19:51:37.326026+0000","last_unstale":"2024-04-12T19:51:38.327881+0000","last_undegraded":"2024-04-12T19:51:38.327881+0000","last_fullsized":"2024-04-12T19:51:38.327881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:15:16.329928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328156+0000","last_change":"2024-04-12T19:51:37.328370+0000","last_active":"2024-04-12T19:51:38.328156+0000","last_peered":"2024-04-12T19:51:38.328156+0000","last_clean":"2024-04-12T19:51:38.328156+0000","last_became_active":"2024-04-12T19:51:37.328016+0000","last_became_peered":"2024-04-12T19:51:37.328016+0000","last_unstale":"2024-04-12T19:51:38.328156+0000","last_undegraded":"2024-04-12T19:51:38.328156+0000","last_fullsized":"2024-04-12T19:51:38.328156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:59:33.416853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.322194+0000","last_change":"2024-04-12T19:51:36.322194+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.322194+0000","last_undegraded":"2024-04-12T19:51:36.322194+0000","last_fullsized":"2024-04-12T19:51:36.322194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.298505+0000","last_change":"2024-04-12T19:51:36.298505+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.298505+0000","last_undegraded":"2024-04-12T19:51:36.298505+0000","last_fullsized":"2024-04-12T19:51:36.298505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305444+0000","last_change":"2024-04-12T19:51:36.305444+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305444+0000","last_undegraded":"2024-04-12T19:51:36.305444+0000","last_fullsized":"2024-04-12T19:51:36.305444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316919+0000","last_change":"2024-04-12T19:51:36.316919+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316919+0000","last_undegraded":"2024-04-12T19:51:36.316919+0000","last_fullsized":"2024-04-12T19:51:36.316919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303515+0000","last_change":"2024-04-12T19:51:36.303515+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303515+0000","last_undegraded":"2024-04-12T19:51:36.303515+0000","last_fullsized":"2024-04-12T19:51:36.303515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305132+0000","last_change":"2024-04-12T19:51:36.305132+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305132+0000","last_undegraded":"2024-04-12T19:51:36.305132+0000","last_fullsized":"2024-04-12T19:51:36.305132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304008+0000","last_change":"2024-04-12T19:51:36.304008+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304008+0000","last_undegraded":"2024-04-12T19:51:36.304008+0000","last_fullsized":"2024-04-12T19:51:36.304008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317199+0000","last_change":"2024-04-12T19:51:36.317199+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317199+0000","last_undegraded":"2024-04-12T19:51:36.317199+0000","last_fullsized":"2024-04-12T19:51:36.317199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306291+0000","last_change":"2024-04-12T19:51:36.306291+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306291+0000","last_undegraded":"2024-04-12T19:51:36.306291+0000","last_fullsized":"2024-04-12T19:51:36.306291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297996+0000","last_change":"2024-04-12T19:51:36.297996+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297996+0000","last_undegraded":"2024-04-12T19:51:36.297996+0000","last_fullsized":"2024-04-12T19:51:36.297996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297423+0000","last_change":"2024-04-12T19:51:36.297423+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297423+0000","last_undegraded":"2024-04-12T19:51:36.297423+0000","last_fullsized":"2024-04-12T19:51:36.297423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301892+0000","last_change":"2024-04-12T19:51:36.301892+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301892+0000","last_undegraded":"2024-04-12T19:51:36.301892+0000","last_fullsized":"2024-04-12T19:51:36.301892+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302517+0000","last_change":"2024-04-12T19:51:36.302517+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302517+0000","last_undegraded":"2024-04-12T19:51:36.302517+0000","last_fullsized":"2024-04-12T19:51:36.302517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.318539+0000","last_change":"2024-04-12T19:51:36.318539+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.318539+0000","last_undegraded":"2024-04-12T19:51:36.318539+0000","last_fullsized":"2024-04-12T19:51:36.318539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.324592+0000","last_change":"2024-04-12T19:51:36.323028+0000","last_active":"2024-04-12T19:51:36.324592+0000","last_peered":"2024-04-12T19:51:36.324592+0000","last_clean":"2024-04-12T19:51:36.324592+0000","last_became_active":"2024-04-12T19:51:36.322694+0000","last_became_peered":"2024-04-12T19:51:36.322694+0000","last_unstale":"2024-04-12T19:51:36.324592+0000","last_undegraded":"2024-04-12T19:51:36.324592+0000","last_fullsized":"2024-04-12T19:51:36.324592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:09.348012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301568+0000","last_change":"2024-04-12T19:51:36.301568+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301568+0000","last_undegraded":"2024-04-12T19:51:36.301568+0000","last_fullsized":"2024-04-12T19:51:36.301568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330065+0000","last_change":"2024-04-12T19:51:36.325298+0000","last_active":"2024-04-12T19:51:36.330065+0000","last_peered":"2024-04-12T19:51:36.330065+0000","last_clean":"2024-04-12T19:51:36.330065+0000","last_became_active":"2024-04-12T19:51:36.324222+0000","last_became_peered":"2024-04-12T19:51:36.324222+0000","last_unstale":"2024-04-12T19:51:36.330065+0000","last_undegraded":"2024-04-12T19:51:36.330065+0000","last_fullsized":"2024-04-12T19:51:36.330065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301690+0000","last_change":"2024-04-12T19:51:36.301690+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301690+0000","last_undegraded":"2024-04-12T19:51:36.301690+0000","last_fullsized":"2024-04-12T19:51:36.301690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328355+0000","last_change":"2024-04-12T19:51:36.331707+0000","last_active":"2024-04-12T19:51:38.328355+0000","last_peered":"2024-04-12T19:51:38.328355+0000","last_clean":"2024-04-12T19:51:38.328355+0000","last_became_active":"2024-04-12T19:51:36.331198+0000","last_became_peered":"2024-04-12T19:51:36.331198+0000","last_unstale":"2024-04-12T19:51:38.328355+0000","last_undegraded":"2024-04-12T19:51:38.328355+0000","last_fullsized":"2024-04-12T19:51:38.328355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:18.552643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305586+0000","last_change":"2024-04-12T19:51:36.305586+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305586+0000","last_undegraded":"2024-04-12T19:51:36.305586+0000","last_fullsized":"2024-04-12T19:51:36.305586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.336724+0000","last_change":"2024-04-12T19:51:36.330056+0000","last_active":"2024-04-12T19:51:36.336724+0000","last_peered":"2024-04-12T19:51:36.336724+0000","last_clean":"2024-04-12T19:51:36.336724+0000","last_became_active":"2024-04-12T19:51:36.329657+0000","last_became_peered":"2024-04-12T19:51:36.329657+0000","last_unstale":"2024-04-12T19:51:36.336724+0000","last_undegraded":"2024-04-12T19:51:36.336724+0000","last_fullsized":"2024-04-12T19:51:36.336724+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308524+0000","last_change":"2024-04-12T19:51:36.308524+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308524+0000","last_undegraded":"2024-04-12T19:51:36.308524+0000","last_fullsized":"2024-04-12T19:51:36.308524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328738+0000","last_change":"2024-04-12T19:51:36.327409+0000","last_active":"2024-04-12T19:51:36.328738+0000","last_peered":"2024-04-12T19:51:36.328738+0000","last_clean":"2024-04-12T19:51:36.328738+0000","last_became_active":"2024-04-12T19:51:36.327107+0000","last_became_peered":"2024-04-12T19:51:36.327107+0000","last_unstale":"2024-04-12T19:51:36.328738+0000","last_undegraded":"2024-04-12T19:51:36.328738+0000","last_fullsized":"2024-04-12T19:51:36.328738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306806+0000","last_change":"2024-04-12T19:51:36.306806+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306806+0000","last_undegraded":"2024-04-12T19:51:36.306806+0000","last_fullsized":"2024-04-12T19:51:36.306806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327470+0000","last_change":"2024-04-12T19:51:36.324064+0000","last_active":"2024-04-12T19:51:36.327470+0000","last_peered":"2024-04-12T19:51:36.327470+0000","last_clean":"2024-04-12T19:51:36.327470+0000","last_became_active":"2024-04-12T19:51:36.323334+0000","last_became_peered":"2024-04-12T19:51:36.323334+0000","last_unstale":"2024-04-12T19:51:36.327470+0000","last_undegraded":"2024-04-12T19:51:36.327470+0000","last_fullsized":"2024-04-12T19:51:36.327470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:48:02.416908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.313566+0000","last_change":"2024-04-12T19:51:36.313566+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.313566+0000","last_undegraded":"2024-04-12T19:51:36.313566+0000","last_fullsized":"2024-04-12T19:51:36.313566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326879+0000","last_change":"2024-04-12T19:51:36.326080+0000","last_active":"2024-04-12T19:51:38.326879+0000","last_peered":"2024-04-12T19:51:38.326879+0000","last_clean":"2024-04-12T19:51:38.326879+0000","last_became_active":"2024-04-12T19:51:36.325817+0000","last_became_peered":"2024-04-12T19:51:36.325817+0000","last_unstale":"2024-04-12T19:51:38.326879+0000","last_undegraded":"2024-04-12T19:51:38.326879+0000","last_fullsized":"2024-04-12T19:51:38.326879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:03.515939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302461+0000","last_change":"2024-04-12T19:51:36.302461+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302461+0000","last_undegraded":"2024-04-12T19:51:36.302461+0000","last_fullsized":"2024-04-12T19:51:36.302461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322424+0000","last_change":"2024-04-12T19:51:36.320855+0000","last_active":"2024-04-12T19:51:36.322424+0000","last_peered":"2024-04-12T19:51:36.322424+0000","last_clean":"2024-04-12T19:51:36.322424+0000","last_became_active":"2024-04-12T19:51:36.320566+0000","last_became_peered":"2024-04-12T19:51:36.320566+0000","last_unstale":"2024-04-12T19:51:36.322424+0000","last_undegraded":"2024-04-12T19:51:36.322424+0000","last_fullsized":"2024-04-12T19:51:36.322424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:43:56.356046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303073+0000","last_change":"2024-04-12T19:51:36.303073+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303073+0000","last_undegraded":"2024-04-12T19:51:36.303073+0000","last_fullsized":"2024-04-12T19:51:36.303073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328522+0000","last_change":"2024-04-12T19:51:36.323202+0000","last_active":"2024-04-12T19:51:36.328522+0000","last_peered":"2024-04-12T19:51:36.328522+0000","last_clean":"2024-04-12T19:51:36.328522+0000","last_became_active":"2024-04-12T19:51:36.322883+0000","last_became_peered":"2024-04-12T19:51:36.322883+0000","last_unstale":"2024-04-12T19:51:36.328522+0000","last_undegraded":"2024-04-12T19:51:36.328522+0000","last_fullsized":"2024-04-12T19:51:36.328522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302674+0000","last_change":"2024-04-12T19:51:36.302674+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302674+0000","last_undegraded":"2024-04-12T19:51:36.302674+0000","last_fullsized":"2024-04-12T19:51:36.302674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332954+0000","last_change":"2024-04-12T19:51:36.327948+0000","last_active":"2024-04-12T19:51:36.332954+0000","last_peered":"2024-04-12T19:51:36.332954+0000","last_clean":"2024-04-12T19:51:36.332954+0000","last_became_active":"2024-04-12T19:51:36.327654+0000","last_became_peered":"2024-04-12T19:51:36.327654+0000","last_unstale":"2024-04-12T19:51:36.332954+0000","last_undegraded":"2024-04-12T19:51:36.332954+0000","last_fullsized":"2024-04-12T19:51:36.332954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:44:36.049413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306597+0000","last_change":"2024-04-12T19:51:36.306597+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306597+0000","last_undegraded":"2024-04-12T19:51:36.306597+0000","last_fullsized":"2024-04-12T19:51:36.306597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.309507+0000","last_change":"2024-04-12T19:51:36.304808+0000","last_active":"2024-04-12T19:51:36.309507+0000","last_peered":"2024-04-12T19:51:36.309507+0000","last_clean":"2024-04-12T19:51:36.309507+0000","last_became_active":"2024-04-12T19:51:36.304470+0000","last_became_peered":"2024-04-12T19:51:36.304470+0000","last_unstale":"2024-04-12T19:51:36.309507+0000","last_undegraded":"2024-04-12T19:51:36.309507+0000","last_fullsized":"2024-04-12T19:51:36.309507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:12.187706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301837+0000","last_change":"2024-04-12T19:51:36.301837+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301837+0000","last_undegraded":"2024-04-12T19:51:36.301837+0000","last_fullsized":"2024-04-12T19:51:36.301837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337157+0000","last_change":"2024-04-12T19:51:36.331502+0000","last_active":"2024-04-12T19:51:36.337157+0000","last_peered":"2024-04-12T19:51:36.337157+0000","last_clean":"2024-04-12T19:51:36.337157+0000","last_became_active":"2024-04-12T19:51:36.329297+0000","last_became_peered":"2024-04-12T19:51:36.329297+0000","last_unstale":"2024-04-12T19:51:36.337157+0000","last_undegraded":"2024-04-12T19:51:36.337157+0000","last_fullsized":"2024-04-12T19:51:36.337157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.320204+0000","last_change":"2024-04-12T19:51:36.320204+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.320204+0000","last_undegraded":"2024-04-12T19:51:36.320204+0000","last_fullsized":"2024-04-12T19:51:36.320204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318782+0000","last_change":"2024-04-12T19:51:36.316118+0000","last_active":"2024-04-12T19:51:36.318782+0000","last_peered":"2024-04-12T19:51:36.318782+0000","last_clean":"2024-04-12T19:51:36.318782+0000","last_became_active":"2024-04-12T19:51:36.315852+0000","last_became_peered":"2024-04-12T19:51:36.315852+0000","last_unstale":"2024-04-12T19:51:36.318782+0000","last_undegraded":"2024-04-12T19:51:36.318782+0000","last_fullsized":"2024-04-12T19:51:36.318782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:57:20.899617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305931+0000","last_change":"2024-04-12T19:51:36.305931+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305931+0000","last_undegraded":"2024-04-12T19:51:36.305931+0000","last_fullsized":"2024-04-12T19:51:36.305931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.325983+0000","last_change":"2024-04-12T19:51:36.320973+0000","last_active":"2024-04-12T19:51:36.325983+0000","last_peered":"2024-04-12T19:51:36.325983+0000","last_clean":"2024-04-12T19:51:36.325983+0000","last_became_active":"2024-04-12T19:51:36.320688+0000","last_became_peered":"2024-04-12T19:51:36.320688+0000","last_unstale":"2024-04-12T19:51:36.325983+0000","last_undegraded":"2024-04-12T19:51:36.325983+0000","last_fullsized":"2024-04-12T19:51:36.325983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302481+0000","last_change":"2024-04-12T19:51:36.302481+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302481+0000","last_undegraded":"2024-04-12T19:51:36.302481+0000","last_fullsized":"2024-04-12T19:51:36.302481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.323574+0000","last_change":"2024-04-12T19:51:36.313661+0000","last_active":"2024-04-12T19:51:36.323574+0000","last_peered":"2024-04-12T19:51:36.323574+0000","last_clean":"2024-04-12T19:51:36.323574+0000","last_became_active":"2024-04-12T19:51:36.313249+0000","last_became_peered":"2024-04-12T19:51:36.313249+0000","last_unstale":"2024-04-12T19:51:36.323574+0000","last_undegraded":"2024-04-12T19:51:36.323574+0000","last_fullsized":"2024-04-12T19:51:36.323574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301377+0000","last_change":"2024-04-12T19:51:36.301377+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301377+0000","last_undegraded":"2024-04-12T19:51:36.301377+0000","last_fullsized":"2024-04-12T19:51:36.301377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326281+0000","last_change":"2024-04-12T19:51:36.324874+0000","last_active":"2024-04-12T19:51:38.326281+0000","last_peered":"2024-04-12T19:51:38.326281+0000","last_clean":"2024-04-12T19:51:38.326281+0000","last_became_active":"2024-04-12T19:51:36.324013+0000","last_became_peered":"2024-04-12T19:51:36.324013+0000","last_unstale":"2024-04-12T19:51:38.326281+0000","last_undegraded":"2024-04-12T19:51:38.326281+0000","last_fullsized":"2024-04-12T19:51:38.326281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:10:23.725348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297465+0000","last_change":"2024-04-12T19:51:36.297465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297465+0000","last_undegraded":"2024-04-12T19:51:36.297465+0000","last_fullsized":"2024-04-12T19:51:36.297465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328103+0000","last_change":"2024-04-12T19:51:36.332183+0000","last_active":"2024-04-12T19:51:38.328103+0000","last_peered":"2024-04-12T19:51:38.328103+0000","last_clean":"2024-04-12T19:51:38.328103+0000","last_became_active":"2024-04-12T19:51:36.331021+0000","last_became_peered":"2024-04-12T19:51:36.331021+0000","last_unstale":"2024-04-12T19:51:38.328103+0000","last_undegraded":"2024-04-12T19:51:38.328103+0000","last_fullsized":"2024-04-12T19:51:38.328103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:29:38.531795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309373+0000","last_change":"2024-04-12T19:51:36.309373+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309373+0000","last_undegraded":"2024-04-12T19:51:36.309373+0000","last_fullsized":"2024-04-12T19:51:36.309373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.312675+0000","last_change":"2024-04-12T19:51:36.311352+0000","last_active":"2024-04-12T19:51:36.312675+0000","last_peered":"2024-04-12T19:51:36.312675+0000","last_clean":"2024-04-12T19:51:36.312675+0000","last_became_active":"2024-04-12T19:51:36.310541+0000","last_became_peered":"2024-04-12T19:51:36.310541+0000","last_unstale":"2024-04-12T19:51:36.312675+0000","last_undegraded":"2024-04-12T19:51:36.312675+0000","last_fullsized":"2024-04-12T19:51:36.312675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.312465+0000","last_change":"2024-04-12T19:51:36.312465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.312465+0000","last_undegraded":"2024-04-12T19:51:36.312465+0000","last_fullsized":"2024-04-12T19:51:36.312465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.321500+0000","last_change":"2024-04-12T19:51:36.320077+0000","last_active":"2024-04-12T19:51:36.321500+0000","last_peered":"2024-04-12T19:51:36.321500+0000","last_clean":"2024-04-12T19:51:36.321500+0000","last_became_active":"2024-04-12T19:51:36.319763+0000","last_became_peered":"2024-04-12T19:51:36.319763+0000","last_unstale":"2024-04-12T19:51:36.321500+0000","last_undegraded":"2024-04-12T19:51:36.321500+0000","last_fullsized":"2024-04-12T19:51:36.321500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:38.560305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328222+0000","last_change":"2024-04-12T19:51:37.318524+0000","last_active":"2024-04-12T19:51:38.328222+0000","last_peered":"2024-04-12T19:51:38.328222+0000","last_clean":"2024-04-12T19:51:38.328222+0000","last_became_active":"2024-04-12T19:51:37.318222+0000","last_became_peered":"2024-04-12T19:51:37.318222+0000","last_unstale":"2024-04-12T19:51:38.328222+0000","last_undegraded":"2024-04-12T19:51:38.328222+0000","last_fullsized":"2024-04-12T19:51:38.328222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:10:43.194581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318050+0000","last_change":"2024-04-12T19:51:36.315549+0000","last_active":"2024-04-12T19:51:36.318050+0000","last_peered":"2024-04-12T19:51:36.318050+0000","last_clean":"2024-04-12T19:51:36.318050+0000","last_became_active":"2024-04-12T19:51:36.314815+0000","last_became_peered":"2024-04-12T19:51:36.314815+0000","last_unstale":"2024-04-12T19:51:36.318050+0000","last_undegraded":"2024-04-12T19:51:36.318050+0000","last_fullsized":"2024-04-12T19:51:36.318050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317974+0000","last_change":"2024-04-12T19:51:36.317974+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317974+0000","last_undegraded":"2024-04-12T19:51:36.317974+0000","last_fullsized":"2024-04-12T19:51:36.317974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332929+0000","last_change":"2024-04-12T19:51:36.327171+0000","last_active":"2024-04-12T19:51:36.332929+0000","last_peered":"2024-04-12T19:51:36.332929+0000","last_clean":"2024-04-12T19:51:36.332929+0000","last_became_active":"2024-04-12T19:51:36.326818+0000","last_became_peered":"2024-04-12T19:51:36.326818+0000","last_unstale":"2024-04-12T19:51:36.332929+0000","last_undegraded":"2024-04-12T19:51:36.332929+0000","last_fullsized":"2024-04-12T19:51:36.332929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316050+0000","last_change":"2024-04-12T19:51:36.316050+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316050+0000","last_undegraded":"2024-04-12T19:51:36.316050+0000","last_fullsized":"2024-04-12T19:51:36.316050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318733+0000","last_change":"2024-04-12T19:51:36.315615+0000","last_active":"2024-04-12T19:51:36.318733+0000","last_peered":"2024-04-12T19:51:36.318733+0000","last_clean":"2024-04-12T19:51:36.318733+0000","last_became_active":"2024-04-12T19:51:36.315325+0000","last_became_peered":"2024-04-12T19:51:36.315325+0000","last_unstale":"2024-04-12T19:51:36.318733+0000","last_undegraded":"2024-04-12T19:51:36.318733+0000","last_fullsized":"2024-04-12T19:51:36.318733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:49.798145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.315697+0000","last_change":"2024-04-12T19:51:36.315697+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.315697+0000","last_undegraded":"2024-04-12T19:51:36.315697+0000","last_fullsized":"2024-04-12T19:51:36.315697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329354+0000","last_change":"2024-04-12T19:51:36.326548+0000","last_active":"2024-04-12T19:51:36.329354+0000","last_peered":"2024-04-12T19:51:36.329354+0000","last_clean":"2024-04-12T19:51:36.329354+0000","last_became_active":"2024-04-12T19:51:36.326046+0000","last_became_peered":"2024-04-12T19:51:36.326046+0000","last_unstale":"2024-04-12T19:51:36.329354+0000","last_undegraded":"2024-04-12T19:51:36.329354+0000","last_fullsized":"2024-04-12T19:51:36.329354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308731+0000","last_change":"2024-04-12T19:51:36.308731+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308731+0000","last_undegraded":"2024-04-12T19:51:36.308731+0000","last_fullsized":"2024-04-12T19:51:36.308731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322237+0000","last_change":"2024-04-12T19:51:36.321134+0000","last_active":"2024-04-12T19:51:36.322237+0000","last_peered":"2024-04-12T19:51:36.322237+0000","last_clean":"2024-04-12T19:51:36.322237+0000","last_became_active":"2024-04-12T19:51:36.320811+0000","last_became_peered":"2024-04-12T19:51:36.320811+0000","last_unstale":"2024-04-12T19:51:36.322237+0000","last_undegraded":"2024-04-12T19:51:36.322237+0000","last_fullsized":"2024-04-12T19:51:36.322237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:16.117219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327966+0000","last_change":"2024-04-12T19:51:37.331351+0000","last_active":"2024-04-12T19:51:38.327966+0000","last_peered":"2024-04-12T19:51:38.327966+0000","last_clean":"2024-04-12T19:51:38.327966+0000","last_became_active":"2024-04-12T19:51:37.330895+0000","last_became_peered":"2024-04-12T19:51:37.330895+0000","last_unstale":"2024-04-12T19:51:38.327966+0000","last_undegraded":"2024-04-12T19:51:38.327966+0000","last_fullsized":"2024-04-12T19:51:38.327966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:44:51.414187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330180+0000","last_change":"2024-04-12T19:51:36.327741+0000","last_active":"2024-04-12T19:51:36.330180+0000","last_peered":"2024-04-12T19:51:36.330180+0000","last_clean":"2024-04-12T19:51:36.330180+0000","last_became_active":"2024-04-12T19:51:36.327226+0000","last_became_peered":"2024-04-12T19:51:36.327226+0000","last_unstale":"2024-04-12T19:51:36.330180+0000","last_undegraded":"2024-04-12T19:51:36.330180+0000","last_fullsized":"2024-04-12T19:51:36.330180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296972+0000","last_change":"2024-04-12T19:51:36.296972+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296972+0000","last_undegraded":"2024-04-12T19:51:36.296972+0000","last_fullsized":"2024-04-12T19:51:36.296972+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329105+0000","last_change":"2024-04-12T19:51:36.324717+0000","last_active":"2024-04-12T19:51:36.329105+0000","last_peered":"2024-04-12T19:51:36.329105+0000","last_clean":"2024-04-12T19:51:36.329105+0000","last_became_active":"2024-04-12T19:51:36.324437+0000","last_became_peered":"2024-04-12T19:51:36.324437+0000","last_unstale":"2024-04-12T19:51:36.329105+0000","last_undegraded":"2024-04-12T19:51:36.329105+0000","last_fullsized":"2024-04-12T19:51:36.329105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328579+0000","last_change":"2024-04-12T19:51:37.316433+0000","last_active":"2024-04-12T19:51:38.328579+0000","last_peered":"2024-04-12T19:51:38.328579+0000","last_clean":"2024-04-12T19:51:38.328579+0000","last_became_active":"2024-04-12T19:51:37.316089+0000","last_became_peered":"2024-04-12T19:51:37.316089+0000","last_unstale":"2024-04-12T19:51:38.328579+0000","last_undegraded":"2024-04-12T19:51:38.328579+0000","last_fullsized":"2024-04-12T19:51:38.328579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:26:38.058351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337134+0000","last_change":"2024-04-12T19:51:36.331177+0000","last_active":"2024-04-12T19:51:36.337134+0000","last_peered":"2024-04-12T19:51:36.337134+0000","last_clean":"2024-04-12T19:51:36.337134+0000","last_became_active":"2024-04-12T19:51:36.329431+0000","last_became_peered":"2024-04-12T19:51:36.329431+0000","last_unstale":"2024-04-12T19:51:36.337134+0000","last_undegraded":"2024-04-12T19:51:36.337134+0000","last_fullsized":"2024-04-12T19:51:36.337134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296734+0000","last_change":"2024-04-12T19:51:36.296734+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296734+0000","last_undegraded":"2024-04-12T19:51:36.296734+0000","last_fullsized":"2024-04-12T19:51:36.296734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327735+0000","last_change":"2024-04-12T19:51:36.321573+0000","last_active":"2024-04-12T19:51:36.327735+0000","last_peered":"2024-04-12T19:51:36.327735+0000","last_clean":"2024-04-12T19:51:36.327735+0000","last_became_active":"2024-04-12T19:51:36.321273+0000","last_became_peered":"2024-04-12T19:51:36.321273+0000","last_unstale":"2024-04-12T19:51:36.327735+0000","last_undegraded":"2024-04-12T19:51:36.327735+0000","last_fullsized":"2024-04-12T19:51:36.327735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304472+0000","last_change":"2024-04-12T19:51:36.304472+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304472+0000","last_undegraded":"2024-04-12T19:51:36.304472+0000","last_fullsized":"2024-04-12T19:51:36.304472+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327327+0000","last_change":"2024-04-12T19:51:36.323335+0000","last_active":"2024-04-12T19:51:36.327327+0000","last_peered":"2024-04-12T19:51:36.327327+0000","last_clean":"2024-04-12T19:51:36.327327+0000","last_became_active":"2024-04-12T19:51:36.323010+0000","last_became_peered":"2024-04-12T19:51:36.323010+0000","last_unstale":"2024-04-12T19:51:36.327327+0000","last_undegraded":"2024-04-12T19:51:36.327327+0000","last_fullsized":"2024-04-12T19:51:36.327327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:40:21.315726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302102+0000","last_change":"2024-04-12T19:51:36.302102+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302102+0000","last_undegraded":"2024-04-12T19:51:36.302102+0000","last_fullsized":"2024-04-12T19:51:36.302102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318537+0000","last_change":"2024-04-12T19:51:36.315742+0000","last_active":"2024-04-12T19:51:36.318537+0000","last_peered":"2024-04-12T19:51:36.318537+0000","last_clean":"2024-04-12T19:51:36.318537+0000","last_became_active":"2024-04-12T19:51:36.315459+0000","last_became_peered":"2024-04-12T19:51:36.315459+0000","last_unstale":"2024-04-12T19:51:36.318537+0000","last_undegraded":"2024-04-12T19:51:36.318537+0000","last_fullsized":"2024-04-12T19:51:36.318537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:40.771292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297673+0000","last_change":"2024-04-12T19:51:36.297673+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297673+0000","last_undegraded":"2024-04-12T19:51:36.297673+0000","last_fullsized":"2024-04-12T19:51:36.297673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329868+0000","last_change":"2024-04-12T19:51:36.324603+0000","last_active":"2024-04-12T19:51:36.329868+0000","last_peered":"2024-04-12T19:51:36.329868+0000","last_clean":"2024-04-12T19:51:36.329868+0000","last_became_active":"2024-04-12T19:51:36.322196+0000","last_became_peered":"2024-04-12T19:51:36.322196+0000","last_unstale":"2024-04-12T19:51:36.329868+0000","last_undegraded":"2024-04-12T19:51:36.329868+0000","last_fullsized":"2024-04-12T19:51:36.329868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296831+0000","last_change":"2024-04-12T19:51:36.296831+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296831+0000","last_undegraded":"2024-04-12T19:51:36.296831+0000","last_fullsized":"2024-04-12T19:51:36.296831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322184+0000","last_change":"2024-04-12T19:51:36.320886+0000","last_active":"2024-04-12T19:51:36.322184+0000","last_peered":"2024-04-12T19:51:36.322184+0000","last_clean":"2024-04-12T19:51:36.322184+0000","last_became_active":"2024-04-12T19:51:36.320612+0000","last_became_peered":"2024-04-12T19:51:36.320612+0000","last_unstale":"2024-04-12T19:51:36.322184+0000","last_undegraded":"2024-04-12T19:51:36.322184+0000","last_fullsized":"2024-04-12T19:51:36.322184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:51.567339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.314747+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:51:36.314747+0000","last_peered":"2024-04-12T19:51:36.314747+0000","last_clean":"2024-04-12T19:51:36.314747+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:51:36.314747+0000","last_undegraded":"2024-04-12T19:51:36.314747+0000","last_fullsized":"2024-04-12T19:51:36.314747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":728765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":138397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":722490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:51:42.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:51:42.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.520+0000 7f7132c00640 1 Processor -- start 2024-04-12T19:51:42.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.521+0000 7f7132c00640 1 -- start start 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.522+0000 7f7132c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f712c0cf110 0x7f712c162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.522+0000 7f7132c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712c0d0090 con 0x7f712c162020 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.522+0000 7f7132c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712c0cfad0 con 0x7f712c0d1c90 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.522+0000 7f7132c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f712c163440 con 0x7f712c0cf110 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.523+0000 7f7131800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f712c162020 0x7f712c0d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54464/0 (socket says 172.21.15.32:54464) 2024-04-12T19:51:42.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.523+0000 7f7131800640 1 -- 172.21.15.32:0/3023073116 learned_addr learned my addr 172.21.15.32:0/3023073116 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.523+0000 7f7132200640 1 --2- 172.21.15.32:0/3023073116 >> v2:172.21.15.169:3300/0 conn(0x7f712c0cf110 0x7f712c162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.524+0000 7f712be00640 1 -- 172.21.15.32:0/3023073116 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162719745 0 0) 0x7f7120001490 con 0x7f712c162020 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.524+0000 7f712be00640 1 -- 172.21.15.32:0/3023073116 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7114003520 con 0x7f712c162020 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.524+0000 7f7132200640 1 -- 172.21.15.32:0/3023073116 >> v1:172.21.15.169:6789/0 conn(0x7f712c0d1c90 legacy=0x7f712c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.524+0000 7f7132200640 1 -- 172.21.15.32:0/3023073116 >> v1:172.21.15.32:6789/0 conn(0x7f712c162020 legacy=0x7f712c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.524+0000 7f7132200640 1 -- 172.21.15.32:0/3023073116 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c0cfdb0 con 0x7f712c0cf110 2024-04-12T19:51:42.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.525+0000 7f7132200640 1 --2- 172.21.15.32:0/3023073116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c0cf110 0x7f712c162f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71240097a0 tx=0x7f712402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812e8c85a95f3f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f712be00640 1 -- 172.21.15.32:0/3023073116 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124034070 con 0x7f712c0cf110 2024-04-12T19:51:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f712be00640 1 -- 172.21.15.32:0/3023073116 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124030610 con 0x7f712c0cf110 2024-04-12T19:51:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f712be00640 1 -- 172.21.15.32:0/3023073116 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240309c0 con 0x7f712c0cf110 2024-04-12T19:51:42.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f7132c00640 1 -- 172.21.15.32:0/3023073116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c0cf110 msgr2=0x7f712c162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f7132c00640 1 --2- 172.21.15.32:0/3023073116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c0cf110 0x7f712c162f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f71240097a0 tx=0x7f712402b960 comp rx=0 tx=0).stop 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f7132c00640 1 -- 172.21.15.32:0/3023073116 shutdown_connections 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f7132c00640 1 --2- 172.21.15.32:0/3023073116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c0cf110 0x7f712c162f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.526+0000 7f7132c00640 1 -- 172.21.15.32:0/3023073116 >> 172.21.15.32:0/3023073116 conn(0x7f712c06cf00 msgr2=0x7f712c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.527+0000 7f7132c00640 1 -- 172.21.15.32:0/3023073116 shutdown_connections 2024-04-12T19:51:42.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.527+0000 7f7132c00640 1 -- 172.21.15.32:0/3023073116 wait complete. 2024-04-12T19:51:42.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.529+0000 7f7132c00640 1 Processor -- start 2024-04-12T19:51:42.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.529+0000 7f7132c00640 1 -- start start 2024-04-12T19:51:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f712c0d1c90 0x7f712c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c162020 0x7f712c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0cf880 con 0x7f712c0cf110 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0b7b30 con 0x7f712c162020 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0d2e70 con 0x7f712c0d1c90 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53088/0 (socket says 172.21.15.32:53088) 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 -- 172.21.15.32:0/1122413501 learned_addr learned my addr 172.21.15.32:0/1122413501 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7130e00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f712c0d1c90 0x7f712c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7132200640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c162020 0x7f712c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 -- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f712c0d1c90 msgr2=0x7f712c158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f712c0d1c90 0x7f712c158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 -- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c162020 msgr2=0x7f712c159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c162020 0x7f712c159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.531+0000 7f7131800640 1 -- 172.21.15.32:0/1122413501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c159d20 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7131800640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f712000cc00 tx=0x7f712000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120017070 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712000ce10 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c0c8d60 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120012620 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.532+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c0c9600 con 0x7f712c0cf110 2024-04-12T19:51:42.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.533+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c0c8f30 con 0x7f712c0cf110 2024-04-12T19:51:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.534+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f712001a070 con 0x7f712c0cf110 2024-04-12T19:51:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.535+0000 7f7129600640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.535+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f712008bb30 con 0x7f712c0cf110 2024-04-12T19:51:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.536+0000 7f7130e00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.536+0000 7f7130e00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7118007fd0 tx=0x7f7118009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.540+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f712c0d0090 con 0x7f712c0cf110 2024-04-12T19:51:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.740+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f712c0c9160 con 0x7f710c06c8f0 2024-04-12T19:51:42.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.750+0000 7f7129600640 1 -- 172.21.15.32:0/1122413501 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247145 (secure 0 0 0) 0x7f71180050d0 con 0x7f710c06c8f0 2024-04-12T19:51:42.752 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:42.753 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 msgr2=0x7f710c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7118007fd0 tx=0x7f7118009040 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 msgr2=0x7f712c1558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f712000cc00 tx=0x7f712000a430 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 shutdown_connections 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f712c162020 0x7f712c159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f712c0d1c90 0x7f712c158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 --2- 172.21.15.32:0/1122413501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f712c0cf110 0x7f712c1558f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 >> 172.21.15.32:0/1122413501 conn(0x7f712c06cf00 msgr2=0x7f712c158050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:42.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 shutdown_connections 2024-04-12T19:51:42.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:42.755+0000 7f7132c00640 1 -- 172.21.15.32:0/1122413501 wait complete. 2024-04-12T19:51:42.772 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-12T19:51:41.649894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236792,"kb_used_data":3976,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737928,"statfs":{"total":773094113280,"available":772851638272,"internally_reserved":0,"allocated":4071424,"data_stored":2269634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001544"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302279+0000","last_change":"2024-04-12T19:51:36.302279+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302279+0000","last_undegraded":"2024-04-12T19:51:36.302279+0000","last_fullsized":"2024-04-12T19:51:36.302279+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314104+0000","last_change":"2024-04-12T19:51:36.314104+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314104+0000","last_undegraded":"2024-04-12T19:51:36.314104+0000","last_fullsized":"2024-04-12T19:51:36.314104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.310776+0000","last_change":"2024-04-12T19:51:36.310776+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.310776+0000","last_undegraded":"2024-04-12T19:51:36.310776+0000","last_fullsized":"2024-04-12T19:51:36.310776+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.307884+0000","last_change":"2024-04-12T19:51:36.307884+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.307884+0000","last_undegraded":"2024-04-12T19:51:36.307884+0000","last_fullsized":"2024-04-12T19:51:36.307884+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.319707+0000","last_change":"2024-04-12T19:51:36.319707+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.319707+0000","last_undegraded":"2024-04-12T19:51:36.319707+0000","last_fullsized":"2024-04-12T19:51:36.319707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327988+0000","last_change":"2024-04-12T19:51:37.335976+0000","last_active":"2024-04-12T19:51:38.327988+0000","last_peered":"2024-04-12T19:51:38.327988+0000","last_clean":"2024-04-12T19:51:38.327988+0000","last_became_active":"2024-04-12T19:51:37.335672+0000","last_became_peered":"2024-04-12T19:51:37.335672+0000","last_unstale":"2024-04-12T19:51:38.327988+0000","last_undegraded":"2024-04-12T19:51:38.327988+0000","last_fullsized":"2024-04-12T19:51:38.327988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:11:45.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303201+0000","last_change":"2024-04-12T19:51:36.303201+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303201+0000","last_undegraded":"2024-04-12T19:51:36.303201+0000","last_fullsized":"2024-04-12T19:51:36.303201+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306771+0000","last_change":"2024-04-12T19:51:36.306771+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306771+0000","last_undegraded":"2024-04-12T19:51:36.306771+0000","last_fullsized":"2024-04-12T19:51:36.306771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309809+0000","last_change":"2024-04-12T19:51:36.309809+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309809+0000","last_undegraded":"2024-04-12T19:51:36.309809+0000","last_fullsized":"2024-04-12T19:51:36.309809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305225+0000","last_change":"2024-04-12T19:51:36.305225+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305225+0000","last_undegraded":"2024-04-12T19:51:36.305225+0000","last_fullsized":"2024-04-12T19:51:36.305225+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304052+0000","last_change":"2024-04-12T19:51:36.304052+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304052+0000","last_undegraded":"2024-04-12T19:51:36.304052+0000","last_fullsized":"2024-04-12T19:51:36.304052+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309056+0000","last_change":"2024-04-12T19:51:36.309056+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309056+0000","last_undegraded":"2024-04-12T19:51:36.309056+0000","last_fullsized":"2024-04-12T19:51:36.309056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.314265+0000","last_change":"2024-04-12T19:51:36.314265+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.314265+0000","last_undegraded":"2024-04-12T19:51:36.314265+0000","last_fullsized":"2024-04-12T19:51:36.314265+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.300858+0000","last_change":"2024-04-12T19:51:36.300858+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.300858+0000","last_undegraded":"2024-04-12T19:51:36.300858+0000","last_fullsized":"2024-04-12T19:51:36.300858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297764+0000","last_change":"2024-04-12T19:51:36.297764+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297764+0000","last_undegraded":"2024-04-12T19:51:36.297764+0000","last_fullsized":"2024-04-12T19:51:36.297764+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297452+0000","last_change":"2024-04-12T19:51:36.297452+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297452+0000","last_undegraded":"2024-04-12T19:51:36.297452+0000","last_fullsized":"2024-04-12T19:51:36.297452+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303096+0000","last_change":"2024-04-12T19:51:36.303096+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303096+0000","last_undegraded":"2024-04-12T19:51:36.303096+0000","last_fullsized":"2024-04-12T19:51:36.303096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327881+0000","last_change":"2024-04-12T19:51:37.326355+0000","last_active":"2024-04-12T19:51:38.327881+0000","last_peered":"2024-04-12T19:51:38.327881+0000","last_clean":"2024-04-12T19:51:38.327881+0000","last_became_active":"2024-04-12T19:51:37.326026+0000","last_became_peered":"2024-04-12T19:51:37.326026+0000","last_unstale":"2024-04-12T19:51:38.327881+0000","last_undegraded":"2024-04-12T19:51:38.327881+0000","last_fullsized":"2024-04-12T19:51:38.327881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:15:16.329928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328156+0000","last_change":"2024-04-12T19:51:37.328370+0000","last_active":"2024-04-12T19:51:38.328156+0000","last_peered":"2024-04-12T19:51:38.328156+0000","last_clean":"2024-04-12T19:51:38.328156+0000","last_became_active":"2024-04-12T19:51:37.328016+0000","last_became_peered":"2024-04-12T19:51:37.328016+0000","last_unstale":"2024-04-12T19:51:38.328156+0000","last_undegraded":"2024-04-12T19:51:38.328156+0000","last_fullsized":"2024-04-12T19:51:38.328156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:59:33.416853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.322194+0000","last_change":"2024-04-12T19:51:36.322194+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.322194+0000","last_undegraded":"2024-04-12T19:51:36.322194+0000","last_fullsized":"2024-04-12T19:51:36.322194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.298505+0000","last_change":"2024-04-12T19:51:36.298505+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.298505+0000","last_undegraded":"2024-04-12T19:51:36.298505+0000","last_fullsized":"2024-04-12T19:51:36.298505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305444+0000","last_change":"2024-04-12T19:51:36.305444+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305444+0000","last_undegraded":"2024-04-12T19:51:36.305444+0000","last_fullsized":"2024-04-12T19:51:36.305444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316919+0000","last_change":"2024-04-12T19:51:36.316919+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316919+0000","last_undegraded":"2024-04-12T19:51:36.316919+0000","last_fullsized":"2024-04-12T19:51:36.316919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303515+0000","last_change":"2024-04-12T19:51:36.303515+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303515+0000","last_undegraded":"2024-04-12T19:51:36.303515+0000","last_fullsized":"2024-04-12T19:51:36.303515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305132+0000","last_change":"2024-04-12T19:51:36.305132+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305132+0000","last_undegraded":"2024-04-12T19:51:36.305132+0000","last_fullsized":"2024-04-12T19:51:36.305132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304008+0000","last_change":"2024-04-12T19:51:36.304008+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304008+0000","last_undegraded":"2024-04-12T19:51:36.304008+0000","last_fullsized":"2024-04-12T19:51:36.304008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317199+0000","last_change":"2024-04-12T19:51:36.317199+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317199+0000","last_undegraded":"2024-04-12T19:51:36.317199+0000","last_fullsized":"2024-04-12T19:51:36.317199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306291+0000","last_change":"2024-04-12T19:51:36.306291+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306291+0000","last_undegraded":"2024-04-12T19:51:36.306291+0000","last_fullsized":"2024-04-12T19:51:36.306291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297996+0000","last_change":"2024-04-12T19:51:36.297996+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297996+0000","last_undegraded":"2024-04-12T19:51:36.297996+0000","last_fullsized":"2024-04-12T19:51:36.297996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297423+0000","last_change":"2024-04-12T19:51:36.297423+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297423+0000","last_undegraded":"2024-04-12T19:51:36.297423+0000","last_fullsized":"2024-04-12T19:51:36.297423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301892+0000","last_change":"2024-04-12T19:51:36.301892+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301892+0000","last_undegraded":"2024-04-12T19:51:36.301892+0000","last_fullsized":"2024-04-12T19:51:36.301892+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302517+0000","last_change":"2024-04-12T19:51:36.302517+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302517+0000","last_undegraded":"2024-04-12T19:51:36.302517+0000","last_fullsized":"2024-04-12T19:51:36.302517+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.318539+0000","last_change":"2024-04-12T19:51:36.318539+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.318539+0000","last_undegraded":"2024-04-12T19:51:36.318539+0000","last_fullsized":"2024-04-12T19:51:36.318539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.324592+0000","last_change":"2024-04-12T19:51:36.323028+0000","last_active":"2024-04-12T19:51:36.324592+0000","last_peered":"2024-04-12T19:51:36.324592+0000","last_clean":"2024-04-12T19:51:36.324592+0000","last_became_active":"2024-04-12T19:51:36.322694+0000","last_became_peered":"2024-04-12T19:51:36.322694+0000","last_unstale":"2024-04-12T19:51:36.324592+0000","last_undegraded":"2024-04-12T19:51:36.324592+0000","last_fullsized":"2024-04-12T19:51:36.324592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:09.348012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301568+0000","last_change":"2024-04-12T19:51:36.301568+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301568+0000","last_undegraded":"2024-04-12T19:51:36.301568+0000","last_fullsized":"2024-04-12T19:51:36.301568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330065+0000","last_change":"2024-04-12T19:51:36.325298+0000","last_active":"2024-04-12T19:51:36.330065+0000","last_peered":"2024-04-12T19:51:36.330065+0000","last_clean":"2024-04-12T19:51:36.330065+0000","last_became_active":"2024-04-12T19:51:36.324222+0000","last_became_peered":"2024-04-12T19:51:36.324222+0000","last_unstale":"2024-04-12T19:51:36.330065+0000","last_undegraded":"2024-04-12T19:51:36.330065+0000","last_fullsized":"2024-04-12T19:51:36.330065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301690+0000","last_change":"2024-04-12T19:51:36.301690+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301690+0000","last_undegraded":"2024-04-12T19:51:36.301690+0000","last_fullsized":"2024-04-12T19:51:36.301690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328355+0000","last_change":"2024-04-12T19:51:36.331707+0000","last_active":"2024-04-12T19:51:38.328355+0000","last_peered":"2024-04-12T19:51:38.328355+0000","last_clean":"2024-04-12T19:51:38.328355+0000","last_became_active":"2024-04-12T19:51:36.331198+0000","last_became_peered":"2024-04-12T19:51:36.331198+0000","last_unstale":"2024-04-12T19:51:38.328355+0000","last_undegraded":"2024-04-12T19:51:38.328355+0000","last_fullsized":"2024-04-12T19:51:38.328355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:18.552643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305586+0000","last_change":"2024-04-12T19:51:36.305586+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305586+0000","last_undegraded":"2024-04-12T19:51:36.305586+0000","last_fullsized":"2024-04-12T19:51:36.305586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.336724+0000","last_change":"2024-04-12T19:51:36.330056+0000","last_active":"2024-04-12T19:51:36.336724+0000","last_peered":"2024-04-12T19:51:36.336724+0000","last_clean":"2024-04-12T19:51:36.336724+0000","last_became_active":"2024-04-12T19:51:36.329657+0000","last_became_peered":"2024-04-12T19:51:36.329657+0000","last_unstale":"2024-04-12T19:51:36.336724+0000","last_undegraded":"2024-04-12T19:51:36.336724+0000","last_fullsized":"2024-04-12T19:51:36.336724+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308524+0000","last_change":"2024-04-12T19:51:36.308524+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308524+0000","last_undegraded":"2024-04-12T19:51:36.308524+0000","last_fullsized":"2024-04-12T19:51:36.308524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328738+0000","last_change":"2024-04-12T19:51:36.327409+0000","last_active":"2024-04-12T19:51:36.328738+0000","last_peered":"2024-04-12T19:51:36.328738+0000","last_clean":"2024-04-12T19:51:36.328738+0000","last_became_active":"2024-04-12T19:51:36.327107+0000","last_became_peered":"2024-04-12T19:51:36.327107+0000","last_unstale":"2024-04-12T19:51:36.328738+0000","last_undegraded":"2024-04-12T19:51:36.328738+0000","last_fullsized":"2024-04-12T19:51:36.328738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306806+0000","last_change":"2024-04-12T19:51:36.306806+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306806+0000","last_undegraded":"2024-04-12T19:51:36.306806+0000","last_fullsized":"2024-04-12T19:51:36.306806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327470+0000","last_change":"2024-04-12T19:51:36.324064+0000","last_active":"2024-04-12T19:51:36.327470+0000","last_peered":"2024-04-12T19:51:36.327470+0000","last_clean":"2024-04-12T19:51:36.327470+0000","last_became_active":"2024-04-12T19:51:36.323334+0000","last_became_peered":"2024-04-12T19:51:36.323334+0000","last_unstale":"2024-04-12T19:51:36.327470+0000","last_undegraded":"2024-04-12T19:51:36.327470+0000","last_fullsized":"2024-04-12T19:51:36.327470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:48:02.416908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.313566+0000","last_change":"2024-04-12T19:51:36.313566+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.313566+0000","last_undegraded":"2024-04-12T19:51:36.313566+0000","last_fullsized":"2024-04-12T19:51:36.313566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326879+0000","last_change":"2024-04-12T19:51:36.326080+0000","last_active":"2024-04-12T19:51:38.326879+0000","last_peered":"2024-04-12T19:51:38.326879+0000","last_clean":"2024-04-12T19:51:38.326879+0000","last_became_active":"2024-04-12T19:51:36.325817+0000","last_became_peered":"2024-04-12T19:51:36.325817+0000","last_unstale":"2024-04-12T19:51:38.326879+0000","last_undegraded":"2024-04-12T19:51:38.326879+0000","last_fullsized":"2024-04-12T19:51:38.326879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:03.515939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302461+0000","last_change":"2024-04-12T19:51:36.302461+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302461+0000","last_undegraded":"2024-04-12T19:51:36.302461+0000","last_fullsized":"2024-04-12T19:51:36.302461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322424+0000","last_change":"2024-04-12T19:51:36.320855+0000","last_active":"2024-04-12T19:51:36.322424+0000","last_peered":"2024-04-12T19:51:36.322424+0000","last_clean":"2024-04-12T19:51:36.322424+0000","last_became_active":"2024-04-12T19:51:36.320566+0000","last_became_peered":"2024-04-12T19:51:36.320566+0000","last_unstale":"2024-04-12T19:51:36.322424+0000","last_undegraded":"2024-04-12T19:51:36.322424+0000","last_fullsized":"2024-04-12T19:51:36.322424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:43:56.356046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.303073+0000","last_change":"2024-04-12T19:51:36.303073+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.303073+0000","last_undegraded":"2024-04-12T19:51:36.303073+0000","last_fullsized":"2024-04-12T19:51:36.303073+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.328522+0000","last_change":"2024-04-12T19:51:36.323202+0000","last_active":"2024-04-12T19:51:36.328522+0000","last_peered":"2024-04-12T19:51:36.328522+0000","last_clean":"2024-04-12T19:51:36.328522+0000","last_became_active":"2024-04-12T19:51:36.322883+0000","last_became_peered":"2024-04-12T19:51:36.322883+0000","last_unstale":"2024-04-12T19:51:36.328522+0000","last_undegraded":"2024-04-12T19:51:36.328522+0000","last_fullsized":"2024-04-12T19:51:36.328522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302674+0000","last_change":"2024-04-12T19:51:36.302674+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302674+0000","last_undegraded":"2024-04-12T19:51:36.302674+0000","last_fullsized":"2024-04-12T19:51:36.302674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332954+0000","last_change":"2024-04-12T19:51:36.327948+0000","last_active":"2024-04-12T19:51:36.332954+0000","last_peered":"2024-04-12T19:51:36.332954+0000","last_clean":"2024-04-12T19:51:36.332954+0000","last_became_active":"2024-04-12T19:51:36.327654+0000","last_became_peered":"2024-04-12T19:51:36.327654+0000","last_unstale":"2024-04-12T19:51:36.332954+0000","last_undegraded":"2024-04-12T19:51:36.332954+0000","last_fullsized":"2024-04-12T19:51:36.332954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:44:36.049413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.306597+0000","last_change":"2024-04-12T19:51:36.306597+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.306597+0000","last_undegraded":"2024-04-12T19:51:36.306597+0000","last_fullsized":"2024-04-12T19:51:36.306597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.309507+0000","last_change":"2024-04-12T19:51:36.304808+0000","last_active":"2024-04-12T19:51:36.309507+0000","last_peered":"2024-04-12T19:51:36.309507+0000","last_clean":"2024-04-12T19:51:36.309507+0000","last_became_active":"2024-04-12T19:51:36.304470+0000","last_became_peered":"2024-04-12T19:51:36.304470+0000","last_unstale":"2024-04-12T19:51:36.309507+0000","last_undegraded":"2024-04-12T19:51:36.309507+0000","last_fullsized":"2024-04-12T19:51:36.309507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:12.187706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301837+0000","last_change":"2024-04-12T19:51:36.301837+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301837+0000","last_undegraded":"2024-04-12T19:51:36.301837+0000","last_fullsized":"2024-04-12T19:51:36.301837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337157+0000","last_change":"2024-04-12T19:51:36.331502+0000","last_active":"2024-04-12T19:51:36.337157+0000","last_peered":"2024-04-12T19:51:36.337157+0000","last_clean":"2024-04-12T19:51:36.337157+0000","last_became_active":"2024-04-12T19:51:36.329297+0000","last_became_peered":"2024-04-12T19:51:36.329297+0000","last_unstale":"2024-04-12T19:51:36.337157+0000","last_undegraded":"2024-04-12T19:51:36.337157+0000","last_fullsized":"2024-04-12T19:51:36.337157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.320204+0000","last_change":"2024-04-12T19:51:36.320204+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.320204+0000","last_undegraded":"2024-04-12T19:51:36.320204+0000","last_fullsized":"2024-04-12T19:51:36.320204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318782+0000","last_change":"2024-04-12T19:51:36.316118+0000","last_active":"2024-04-12T19:51:36.318782+0000","last_peered":"2024-04-12T19:51:36.318782+0000","last_clean":"2024-04-12T19:51:36.318782+0000","last_became_active":"2024-04-12T19:51:36.315852+0000","last_became_peered":"2024-04-12T19:51:36.315852+0000","last_unstale":"2024-04-12T19:51:36.318782+0000","last_undegraded":"2024-04-12T19:51:36.318782+0000","last_fullsized":"2024-04-12T19:51:36.318782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:57:20.899617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.305931+0000","last_change":"2024-04-12T19:51:36.305931+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.305931+0000","last_undegraded":"2024-04-12T19:51:36.305931+0000","last_fullsized":"2024-04-12T19:51:36.305931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.325983+0000","last_change":"2024-04-12T19:51:36.320973+0000","last_active":"2024-04-12T19:51:36.325983+0000","last_peered":"2024-04-12T19:51:36.325983+0000","last_clean":"2024-04-12T19:51:36.325983+0000","last_became_active":"2024-04-12T19:51:36.320688+0000","last_became_peered":"2024-04-12T19:51:36.320688+0000","last_unstale":"2024-04-12T19:51:36.325983+0000","last_undegraded":"2024-04-12T19:51:36.325983+0000","last_fullsized":"2024-04-12T19:51:36.325983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302481+0000","last_change":"2024-04-12T19:51:36.302481+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302481+0000","last_undegraded":"2024-04-12T19:51:36.302481+0000","last_fullsized":"2024-04-12T19:51:36.302481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.323574+0000","last_change":"2024-04-12T19:51:36.313661+0000","last_active":"2024-04-12T19:51:36.323574+0000","last_peered":"2024-04-12T19:51:36.323574+0000","last_clean":"2024-04-12T19:51:36.323574+0000","last_became_active":"2024-04-12T19:51:36.313249+0000","last_became_peered":"2024-04-12T19:51:36.313249+0000","last_unstale":"2024-04-12T19:51:36.323574+0000","last_undegraded":"2024-04-12T19:51:36.323574+0000","last_fullsized":"2024-04-12T19:51:36.323574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.301377+0000","last_change":"2024-04-12T19:51:36.301377+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.301377+0000","last_undegraded":"2024-04-12T19:51:36.301377+0000","last_fullsized":"2024-04-12T19:51:36.301377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326281+0000","last_change":"2024-04-12T19:51:36.324874+0000","last_active":"2024-04-12T19:51:38.326281+0000","last_peered":"2024-04-12T19:51:38.326281+0000","last_clean":"2024-04-12T19:51:38.326281+0000","last_became_active":"2024-04-12T19:51:36.324013+0000","last_became_peered":"2024-04-12T19:51:36.324013+0000","last_unstale":"2024-04-12T19:51:38.326281+0000","last_undegraded":"2024-04-12T19:51:38.326281+0000","last_fullsized":"2024-04-12T19:51:38.326281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:10:23.725348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297465+0000","last_change":"2024-04-12T19:51:36.297465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297465+0000","last_undegraded":"2024-04-12T19:51:36.297465+0000","last_fullsized":"2024-04-12T19:51:36.297465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328103+0000","last_change":"2024-04-12T19:51:36.332183+0000","last_active":"2024-04-12T19:51:38.328103+0000","last_peered":"2024-04-12T19:51:38.328103+0000","last_clean":"2024-04-12T19:51:38.328103+0000","last_became_active":"2024-04-12T19:51:36.331021+0000","last_became_peered":"2024-04-12T19:51:36.331021+0000","last_unstale":"2024-04-12T19:51:38.328103+0000","last_undegraded":"2024-04-12T19:51:38.328103+0000","last_fullsized":"2024-04-12T19:51:38.328103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:29:38.531795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.309373+0000","last_change":"2024-04-12T19:51:36.309373+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.309373+0000","last_undegraded":"2024-04-12T19:51:36.309373+0000","last_fullsized":"2024-04-12T19:51:36.309373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.312675+0000","last_change":"2024-04-12T19:51:36.311352+0000","last_active":"2024-04-12T19:51:36.312675+0000","last_peered":"2024-04-12T19:51:36.312675+0000","last_clean":"2024-04-12T19:51:36.312675+0000","last_became_active":"2024-04-12T19:51:36.310541+0000","last_became_peered":"2024-04-12T19:51:36.310541+0000","last_unstale":"2024-04-12T19:51:36.312675+0000","last_undegraded":"2024-04-12T19:51:36.312675+0000","last_fullsized":"2024-04-12T19:51:36.312675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.312465+0000","last_change":"2024-04-12T19:51:36.312465+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.312465+0000","last_undegraded":"2024-04-12T19:51:36.312465+0000","last_fullsized":"2024-04-12T19:51:36.312465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.321500+0000","last_change":"2024-04-12T19:51:36.320077+0000","last_active":"2024-04-12T19:51:36.321500+0000","last_peered":"2024-04-12T19:51:36.321500+0000","last_clean":"2024-04-12T19:51:36.321500+0000","last_became_active":"2024-04-12T19:51:36.319763+0000","last_became_peered":"2024-04-12T19:51:36.319763+0000","last_unstale":"2024-04-12T19:51:36.321500+0000","last_undegraded":"2024-04-12T19:51:36.321500+0000","last_fullsized":"2024-04-12T19:51:36.321500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:38.560305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328222+0000","last_change":"2024-04-12T19:51:37.318524+0000","last_active":"2024-04-12T19:51:38.328222+0000","last_peered":"2024-04-12T19:51:38.328222+0000","last_clean":"2024-04-12T19:51:38.328222+0000","last_became_active":"2024-04-12T19:51:37.318222+0000","last_became_peered":"2024-04-12T19:51:37.318222+0000","last_unstale":"2024-04-12T19:51:38.328222+0000","last_undegraded":"2024-04-12T19:51:38.328222+0000","last_fullsized":"2024-04-12T19:51:38.328222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:10:43.194581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318050+0000","last_change":"2024-04-12T19:51:36.315549+0000","last_active":"2024-04-12T19:51:36.318050+0000","last_peered":"2024-04-12T19:51:36.318050+0000","last_clean":"2024-04-12T19:51:36.318050+0000","last_became_active":"2024-04-12T19:51:36.314815+0000","last_became_peered":"2024-04-12T19:51:36.314815+0000","last_unstale":"2024-04-12T19:51:36.318050+0000","last_undegraded":"2024-04-12T19:51:36.318050+0000","last_fullsized":"2024-04-12T19:51:36.318050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.317974+0000","last_change":"2024-04-12T19:51:36.317974+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.317974+0000","last_undegraded":"2024-04-12T19:51:36.317974+0000","last_fullsized":"2024-04-12T19:51:36.317974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.332929+0000","last_change":"2024-04-12T19:51:36.327171+0000","last_active":"2024-04-12T19:51:36.332929+0000","last_peered":"2024-04-12T19:51:36.332929+0000","last_clean":"2024-04-12T19:51:36.332929+0000","last_became_active":"2024-04-12T19:51:36.326818+0000","last_became_peered":"2024-04-12T19:51:36.326818+0000","last_unstale":"2024-04-12T19:51:36.332929+0000","last_undegraded":"2024-04-12T19:51:36.332929+0000","last_fullsized":"2024-04-12T19:51:36.332929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.316050+0000","last_change":"2024-04-12T19:51:36.316050+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.316050+0000","last_undegraded":"2024-04-12T19:51:36.316050+0000","last_fullsized":"2024-04-12T19:51:36.316050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318733+0000","last_change":"2024-04-12T19:51:36.315615+0000","last_active":"2024-04-12T19:51:36.318733+0000","last_peered":"2024-04-12T19:51:36.318733+0000","last_clean":"2024-04-12T19:51:36.318733+0000","last_became_active":"2024-04-12T19:51:36.315325+0000","last_became_peered":"2024-04-12T19:51:36.315325+0000","last_unstale":"2024-04-12T19:51:36.318733+0000","last_undegraded":"2024-04-12T19:51:36.318733+0000","last_fullsized":"2024-04-12T19:51:36.318733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:49.798145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.315697+0000","last_change":"2024-04-12T19:51:36.315697+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.315697+0000","last_undegraded":"2024-04-12T19:51:36.315697+0000","last_fullsized":"2024-04-12T19:51:36.315697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329354+0000","last_change":"2024-04-12T19:51:36.326548+0000","last_active":"2024-04-12T19:51:36.329354+0000","last_peered":"2024-04-12T19:51:36.329354+0000","last_clean":"2024-04-12T19:51:36.329354+0000","last_became_active":"2024-04-12T19:51:36.326046+0000","last_became_peered":"2024-04-12T19:51:36.326046+0000","last_unstale":"2024-04-12T19:51:36.329354+0000","last_undegraded":"2024-04-12T19:51:36.329354+0000","last_fullsized":"2024-04-12T19:51:36.329354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:18:18.932006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.308731+0000","last_change":"2024-04-12T19:51:36.308731+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.308731+0000","last_undegraded":"2024-04-12T19:51:36.308731+0000","last_fullsized":"2024-04-12T19:51:36.308731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322237+0000","last_change":"2024-04-12T19:51:36.321134+0000","last_active":"2024-04-12T19:51:36.322237+0000","last_peered":"2024-04-12T19:51:36.322237+0000","last_clean":"2024-04-12T19:51:36.322237+0000","last_became_active":"2024-04-12T19:51:36.320811+0000","last_became_peered":"2024-04-12T19:51:36.320811+0000","last_unstale":"2024-04-12T19:51:36.322237+0000","last_undegraded":"2024-04-12T19:51:36.322237+0000","last_fullsized":"2024-04-12T19:51:36.322237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:16.117219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327966+0000","last_change":"2024-04-12T19:51:37.331351+0000","last_active":"2024-04-12T19:51:38.327966+0000","last_peered":"2024-04-12T19:51:38.327966+0000","last_clean":"2024-04-12T19:51:38.327966+0000","last_became_active":"2024-04-12T19:51:37.330895+0000","last_became_peered":"2024-04-12T19:51:37.330895+0000","last_unstale":"2024-04-12T19:51:38.327966+0000","last_undegraded":"2024-04-12T19:51:38.327966+0000","last_fullsized":"2024-04-12T19:51:38.327966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:44:51.414187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.330180+0000","last_change":"2024-04-12T19:51:36.327741+0000","last_active":"2024-04-12T19:51:36.330180+0000","last_peered":"2024-04-12T19:51:36.330180+0000","last_clean":"2024-04-12T19:51:36.330180+0000","last_became_active":"2024-04-12T19:51:36.327226+0000","last_became_peered":"2024-04-12T19:51:36.327226+0000","last_unstale":"2024-04-12T19:51:36.330180+0000","last_undegraded":"2024-04-12T19:51:36.330180+0000","last_fullsized":"2024-04-12T19:51:36.330180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296972+0000","last_change":"2024-04-12T19:51:36.296972+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296972+0000","last_undegraded":"2024-04-12T19:51:36.296972+0000","last_fullsized":"2024-04-12T19:51:36.296972+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329105+0000","last_change":"2024-04-12T19:51:36.324717+0000","last_active":"2024-04-12T19:51:36.329105+0000","last_peered":"2024-04-12T19:51:36.329105+0000","last_clean":"2024-04-12T19:51:36.329105+0000","last_became_active":"2024-04-12T19:51:36.324437+0000","last_became_peered":"2024-04-12T19:51:36.324437+0000","last_unstale":"2024-04-12T19:51:36.329105+0000","last_undegraded":"2024-04-12T19:51:36.329105+0000","last_fullsized":"2024-04-12T19:51:36.329105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328579+0000","last_change":"2024-04-12T19:51:37.316433+0000","last_active":"2024-04-12T19:51:38.328579+0000","last_peered":"2024-04-12T19:51:38.328579+0000","last_clean":"2024-04-12T19:51:38.328579+0000","last_became_active":"2024-04-12T19:51:37.316089+0000","last_became_peered":"2024-04-12T19:51:37.316089+0000","last_unstale":"2024-04-12T19:51:38.328579+0000","last_undegraded":"2024-04-12T19:51:38.328579+0000","last_fullsized":"2024-04-12T19:51:38.328579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:26:38.058351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.337134+0000","last_change":"2024-04-12T19:51:36.331177+0000","last_active":"2024-04-12T19:51:36.337134+0000","last_peered":"2024-04-12T19:51:36.337134+0000","last_clean":"2024-04-12T19:51:36.337134+0000","last_became_active":"2024-04-12T19:51:36.329431+0000","last_became_peered":"2024-04-12T19:51:36.329431+0000","last_unstale":"2024-04-12T19:51:36.337134+0000","last_undegraded":"2024-04-12T19:51:36.337134+0000","last_fullsized":"2024-04-12T19:51:36.337134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296734+0000","last_change":"2024-04-12T19:51:36.296734+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296734+0000","last_undegraded":"2024-04-12T19:51:36.296734+0000","last_fullsized":"2024-04-12T19:51:36.296734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327735+0000","last_change":"2024-04-12T19:51:36.321573+0000","last_active":"2024-04-12T19:51:36.327735+0000","last_peered":"2024-04-12T19:51:36.327735+0000","last_clean":"2024-04-12T19:51:36.327735+0000","last_became_active":"2024-04-12T19:51:36.321273+0000","last_became_peered":"2024-04-12T19:51:36.321273+0000","last_unstale":"2024-04-12T19:51:36.327735+0000","last_undegraded":"2024-04-12T19:51:36.327735+0000","last_fullsized":"2024-04-12T19:51:36.327735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:50:38.145954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.304472+0000","last_change":"2024-04-12T19:51:36.304472+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.304472+0000","last_undegraded":"2024-04-12T19:51:36.304472+0000","last_fullsized":"2024-04-12T19:51:36.304472+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.327327+0000","last_change":"2024-04-12T19:51:36.323335+0000","last_active":"2024-04-12T19:51:36.327327+0000","last_peered":"2024-04-12T19:51:36.327327+0000","last_clean":"2024-04-12T19:51:36.327327+0000","last_became_active":"2024-04-12T19:51:36.323010+0000","last_became_peered":"2024-04-12T19:51:36.323010+0000","last_unstale":"2024-04-12T19:51:36.327327+0000","last_undegraded":"2024-04-12T19:51:36.327327+0000","last_fullsized":"2024-04-12T19:51:36.327327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:40:21.315726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.302102+0000","last_change":"2024-04-12T19:51:36.302102+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.302102+0000","last_undegraded":"2024-04-12T19:51:36.302102+0000","last_fullsized":"2024-04-12T19:51:36.302102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.318537+0000","last_change":"2024-04-12T19:51:36.315742+0000","last_active":"2024-04-12T19:51:36.318537+0000","last_peered":"2024-04-12T19:51:36.318537+0000","last_clean":"2024-04-12T19:51:36.318537+0000","last_became_active":"2024-04-12T19:51:36.315459+0000","last_became_peered":"2024-04-12T19:51:36.315459+0000","last_unstale":"2024-04-12T19:51:36.318537+0000","last_undegraded":"2024-04-12T19:51:36.318537+0000","last_fullsized":"2024-04-12T19:51:36.318537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:42:40.771292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.297673+0000","last_change":"2024-04-12T19:51:36.297673+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.297673+0000","last_undegraded":"2024-04-12T19:51:36.297673+0000","last_fullsized":"2024-04-12T19:51:36.297673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.329868+0000","last_change":"2024-04-12T19:51:36.324603+0000","last_active":"2024-04-12T19:51:36.329868+0000","last_peered":"2024-04-12T19:51:36.329868+0000","last_clean":"2024-04-12T19:51:36.329868+0000","last_became_active":"2024-04-12T19:51:36.322196+0000","last_became_peered":"2024-04-12T19:51:36.322196+0000","last_unstale":"2024-04-12T19:51:36.329868+0000","last_undegraded":"2024-04-12T19:51:36.329868+0000","last_fullsized":"2024-04-12T19:51:36.329868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-04-12T19:51:36.296831+0000","last_change":"2024-04-12T19:51:36.296831+0000","last_active":"2024-04-12T19:51:36.272283+0000","last_peered":"2024-04-12T19:51:36.272283+0000","last_clean":"2024-04-12T19:51:36.272283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-12T19:51:36.296831+0000","last_undegraded":"2024-04-12T19:51:36.296831+0000","last_fullsized":"2024-04-12T19:51:36.296831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.322184+0000","last_change":"2024-04-12T19:51:36.320886+0000","last_active":"2024-04-12T19:51:36.322184+0000","last_peered":"2024-04-12T19:51:36.322184+0000","last_clean":"2024-04-12T19:51:36.322184+0000","last_became_active":"2024-04-12T19:51:36.320612+0000","last_became_peered":"2024-04-12T19:51:36.320612+0000","last_unstale":"2024-04-12T19:51:36.322184+0000","last_undegraded":"2024-04-12T19:51:36.322184+0000","last_fullsized":"2024-04-12T19:51:36.322184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:51.567339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-12T19:51:36.314747+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:51:36.314747+0000","last_peered":"2024-04-12T19:51:36.314747+0000","last_clean":"2024-04-12T19:51:36.314747+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:51:36.314747+0000","last_undegraded":"2024-04-12T19:51:36.314747+0000","last_fullsized":"2024-04-12T19:51:36.314747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":728765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":138397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":722490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":132122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:51:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-12T19:51:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302279+0000', 'last_change': '2024-04-12T19:51:36.302279+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302279+0000', 'last_undegraded': '2024-04-12T19:51:36.302279+0000', 'last_fullsized': '2024-04-12T19:51:36.302279+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-12T19:51:42.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.314104+0000', 'last_change': '2024-04-12T19:51:36.314104+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.314104+0000', 'last_undegraded': '2024-04-12T19:51:36.314104+0000', 'last_fullsized': '2024-04-12T19:51:36.314104+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.310776+0000', 'last_change': '2024-04-12T19:51:36.310776+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.310776+0000', 'last_undegraded': '2024-04-12T19:51:36.310776+0000', 'last_fullsized': '2024-04-12T19:51:36.310776+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.307884+0000', 'last_change': '2024-04-12T19:51:36.307884+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.307884+0000', 'last_undegraded': '2024-04-12T19:51:36.307884+0000', 'last_fullsized': '2024-04-12T19:51:36.307884+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.319707+0000', 'last_change': '2024-04-12T19:51:36.319707+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.319707+0000', 'last_undegraded': '2024-04-12T19:51:36.319707+0000', 'last_fullsized': '2024-04-12T19:51:36.319707+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.303201+0000', 'last_change': '2024-04-12T19:51:36.303201+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.303201+0000', 'last_undegraded': '2024-04-12T19:51:36.303201+0000', 'last_fullsized': '2024-04-12T19:51:36.303201+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.306771+0000', 'last_change': '2024-04-12T19:51:36.306771+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.306771+0000', 'last_undegraded': '2024-04-12T19:51:36.306771+0000', 'last_fullsized': '2024-04-12T19:51:36.306771+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.309809+0000', 'last_change': '2024-04-12T19:51:36.309809+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.309809+0000', 'last_undegraded': '2024-04-12T19:51:36.309809+0000', 'last_fullsized': '2024-04-12T19:51:36.309809+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.305225+0000', 'last_change': '2024-04-12T19:51:36.305225+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.305225+0000', 'last_undegraded': '2024-04-12T19:51:36.305225+0000', 'last_fullsized': '2024-04-12T19:51:36.305225+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.304052+0000', 'last_change': '2024-04-12T19:51:36.304052+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.304052+0000', 'last_undegraded': '2024-04-12T19:51:36.304052+0000', 'last_fullsized': '2024-04-12T19:51:36.304052+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.309056+0000', 'last_change': '2024-04-12T19:51:36.309056+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.309056+0000', 'last_undegraded': '2024-04-12T19:51:36.309056+0000', 'last_fullsized': '2024-04-12T19:51:36.309056+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.314265+0000', 'last_change': '2024-04-12T19:51:36.314265+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.314265+0000', 'last_undegraded': '2024-04-12T19:51:36.314265+0000', 'last_fullsized': '2024-04-12T19:51:36.314265+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.300858+0000', 'last_change': '2024-04-12T19:51:36.300858+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.300858+0000', 'last_undegraded': '2024-04-12T19:51:36.300858+0000', 'last_fullsized': '2024-04-12T19:51:36.300858+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297764+0000', 'last_change': '2024-04-12T19:51:36.297764+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297764+0000', 'last_undegraded': '2024-04-12T19:51:36.297764+0000', 'last_fullsized': '2024-04-12T19:51:36.297764+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297452+0000', 'last_change': '2024-04-12T19:51:36.297452+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297452+0000', 'last_undegraded': '2024-04-12T19:51:36.297452+0000', 'last_fullsized': '2024-04-12T19:51:36.297452+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.303096+0000', 'last_change': '2024-04-12T19:51:36.303096+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.303096+0000', 'last_undegraded': '2024-04-12T19:51:36.303096+0000', 'last_fullsized': '2024-04-12T19:51:36.303096+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.322194+0000', 'last_change': '2024-04-12T19:51:36.322194+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.322194+0000', 'last_undegraded': '2024-04-12T19:51:36.322194+0000', 'last_fullsized': '2024-04-12T19:51:36.322194+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.298505+0000', 'last_change': '2024-04-12T19:51:36.298505+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.298505+0000', 'last_undegraded': '2024-04-12T19:51:36.298505+0000', 'last_fullsized': '2024-04-12T19:51:36.298505+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.305444+0000', 'last_change': '2024-04-12T19:51:36.305444+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.305444+0000', 'last_undegraded': '2024-04-12T19:51:36.305444+0000', 'last_fullsized': '2024-04-12T19:51:36.305444+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.316919+0000', 'last_change': '2024-04-12T19:51:36.316919+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.316919+0000', 'last_undegraded': '2024-04-12T19:51:36.316919+0000', 'last_fullsized': '2024-04-12T19:51:36.316919+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.303515+0000', 'last_change': '2024-04-12T19:51:36.303515+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.303515+0000', 'last_undegraded': '2024-04-12T19:51:36.303515+0000', 'last_fullsized': '2024-04-12T19:51:36.303515+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.305132+0000', 'last_change': '2024-04-12T19:51:36.305132+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.305132+0000', 'last_undegraded': '2024-04-12T19:51:36.305132+0000', 'last_fullsized': '2024-04-12T19:51:36.305132+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.304008+0000', 'last_change': '2024-04-12T19:51:36.304008+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.304008+0000', 'last_undegraded': '2024-04-12T19:51:36.304008+0000', 'last_fullsized': '2024-04-12T19:51:36.304008+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.317199+0000', 'last_change': '2024-04-12T19:51:36.317199+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.317199+0000', 'last_undegraded': '2024-04-12T19:51:36.317199+0000', 'last_fullsized': '2024-04-12T19:51:36.317199+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.306291+0000', 'last_change': '2024-04-12T19:51:36.306291+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.306291+0000', 'last_undegraded': '2024-04-12T19:51:36.306291+0000', 'last_fullsized': '2024-04-12T19:51:36.306291+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-12T19:51:42.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297996+0000', 'last_change': '2024-04-12T19:51:36.297996+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297996+0000', 'last_undegraded': '2024-04-12T19:51:36.297996+0000', 'last_fullsized': '2024-04-12T19:51:36.297996+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297423+0000', 'last_change': '2024-04-12T19:51:36.297423+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297423+0000', 'last_undegraded': '2024-04-12T19:51:36.297423+0000', 'last_fullsized': '2024-04-12T19:51:36.297423+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.301892+0000', 'last_change': '2024-04-12T19:51:36.301892+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.301892+0000', 'last_undegraded': '2024-04-12T19:51:36.301892+0000', 'last_fullsized': '2024-04-12T19:51:36.301892+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302517+0000', 'last_change': '2024-04-12T19:51:36.302517+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302517+0000', 'last_undegraded': '2024-04-12T19:51:36.302517+0000', 'last_fullsized': '2024-04-12T19:51:36.302517+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.318539+0000', 'last_change': '2024-04-12T19:51:36.318539+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.318539+0000', 'last_undegraded': '2024-04-12T19:51:36.318539+0000', 'last_fullsized': '2024-04-12T19:51:36.318539+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.301568+0000', 'last_change': '2024-04-12T19:51:36.301568+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.301568+0000', 'last_undegraded': '2024-04-12T19:51:36.301568+0000', 'last_fullsized': '2024-04-12T19:51:36.301568+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.301690+0000', 'last_change': '2024-04-12T19:51:36.301690+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.301690+0000', 'last_undegraded': '2024-04-12T19:51:36.301690+0000', 'last_fullsized': '2024-04-12T19:51:36.301690+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.305586+0000', 'last_change': '2024-04-12T19:51:36.305586+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.305586+0000', 'last_undegraded': '2024-04-12T19:51:36.305586+0000', 'last_fullsized': '2024-04-12T19:51:36.305586+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.308524+0000', 'last_change': '2024-04-12T19:51:36.308524+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.308524+0000', 'last_undegraded': '2024-04-12T19:51:36.308524+0000', 'last_fullsized': '2024-04-12T19:51:36.308524+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.306806+0000', 'last_change': '2024-04-12T19:51:36.306806+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.306806+0000', 'last_undegraded': '2024-04-12T19:51:36.306806+0000', 'last_fullsized': '2024-04-12T19:51:36.306806+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.313566+0000', 'last_change': '2024-04-12T19:51:36.313566+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.313566+0000', 'last_undegraded': '2024-04-12T19:51:36.313566+0000', 'last_fullsized': '2024-04-12T19:51:36.313566+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-12T19:51:42.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302461+0000', 'last_change': '2024-04-12T19:51:36.302461+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302461+0000', 'last_undegraded': '2024-04-12T19:51:36.302461+0000', 'last_fullsized': '2024-04-12T19:51:36.302461+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.303073+0000', 'last_change': '2024-04-12T19:51:36.303073+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.303073+0000', 'last_undegraded': '2024-04-12T19:51:36.303073+0000', 'last_fullsized': '2024-04-12T19:51:36.303073+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302674+0000', 'last_change': '2024-04-12T19:51:36.302674+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302674+0000', 'last_undegraded': '2024-04-12T19:51:36.302674+0000', 'last_fullsized': '2024-04-12T19:51:36.302674+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.306597+0000', 'last_change': '2024-04-12T19:51:36.306597+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.306597+0000', 'last_undegraded': '2024-04-12T19:51:36.306597+0000', 'last_fullsized': '2024-04-12T19:51:36.306597+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.301837+0000', 'last_change': '2024-04-12T19:51:36.301837+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.301837+0000', 'last_undegraded': '2024-04-12T19:51:36.301837+0000', 'last_fullsized': '2024-04-12T19:51:36.301837+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.320204+0000', 'last_change': '2024-04-12T19:51:36.320204+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.320204+0000', 'last_undegraded': '2024-04-12T19:51:36.320204+0000', 'last_fullsized': '2024-04-12T19:51:36.320204+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.305931+0000', 'last_change': '2024-04-12T19:51:36.305931+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.305931+0000', 'last_undegraded': '2024-04-12T19:51:36.305931+0000', 'last_fullsized': '2024-04-12T19:51:36.305931+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302481+0000', 'last_change': '2024-04-12T19:51:36.302481+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302481+0000', 'last_undegraded': '2024-04-12T19:51:36.302481+0000', 'last_fullsized': '2024-04-12T19:51:36.302481+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.301377+0000', 'last_change': '2024-04-12T19:51:36.301377+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.301377+0000', 'last_undegraded': '2024-04-12T19:51:36.301377+0000', 'last_fullsized': '2024-04-12T19:51:36.301377+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297465+0000', 'last_change': '2024-04-12T19:51:36.297465+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297465+0000', 'last_undegraded': '2024-04-12T19:51:36.297465+0000', 'last_fullsized': '2024-04-12T19:51:36.297465+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.309373+0000', 'last_change': '2024-04-12T19:51:36.309373+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.309373+0000', 'last_undegraded': '2024-04-12T19:51:36.309373+0000', 'last_fullsized': '2024-04-12T19:51:36.309373+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.312465+0000', 'last_change': '2024-04-12T19:51:36.312465+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.312465+0000', 'last_undegraded': '2024-04-12T19:51:36.312465+0000', 'last_fullsized': '2024-04-12T19:51:36.312465+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-12T19:51:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.317974+0000', 'last_change': '2024-04-12T19:51:36.317974+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.317974+0000', 'last_undegraded': '2024-04-12T19:51:36.317974+0000', 'last_fullsized': '2024-04-12T19:51:36.317974+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.316050+0000', 'last_change': '2024-04-12T19:51:36.316050+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.316050+0000', 'last_undegraded': '2024-04-12T19:51:36.316050+0000', 'last_fullsized': '2024-04-12T19:51:36.316050+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.315697+0000', 'last_change': '2024-04-12T19:51:36.315697+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.315697+0000', 'last_undegraded': '2024-04-12T19:51:36.315697+0000', 'last_fullsized': '2024-04-12T19:51:36.315697+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.308731+0000', 'last_change': '2024-04-12T19:51:36.308731+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.308731+0000', 'last_undegraded': '2024-04-12T19:51:36.308731+0000', 'last_fullsized': '2024-04-12T19:51:36.308731+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.296972+0000', 'last_change': '2024-04-12T19:51:36.296972+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.296972+0000', 'last_undegraded': '2024-04-12T19:51:36.296972+0000', 'last_fullsized': '2024-04-12T19:51:36.296972+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-12T19:51:42.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.296734+0000', 'last_change': '2024-04-12T19:51:36.296734+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.296734+0000', 'last_undegraded': '2024-04-12T19:51:36.296734+0000', 'last_fullsized': '2024-04-12T19:51:36.296734+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.304472+0000', 'last_change': '2024-04-12T19:51:36.304472+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.304472+0000', 'last_undegraded': '2024-04-12T19:51:36.304472+0000', 'last_fullsized': '2024-04-12T19:51:36.304472+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.302102+0000', 'last_change': '2024-04-12T19:51:36.302102+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.302102+0000', 'last_undegraded': '2024-04-12T19:51:36.302102+0000', 'last_fullsized': '2024-04-12T19:51:36.302102+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.297673+0000', 'last_change': '2024-04-12T19:51:36.297673+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.297673+0000', 'last_undegraded': '2024-04-12T19:51:36.297673+0000', 'last_fullsized': '2024-04-12T19:51:36.297673+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-12T19:51:42.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-04-12T19:51:36.296831+0000', 'last_change': '2024-04-12T19:51:36.296831+0000', 'last_active': '2024-04-12T19:51:36.272283+0000', 'last_peered': '2024-04-12T19:51:36.272283+0000', 'last_clean': '2024-04-12T19:51:36.272283+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-12T19:51:36.296831+0000', 'last_undegraded': '2024-04-12T19:51:36.296831+0000', 'last_fullsized': '2024-04-12T19:51:36.296831+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'last_clean_scrub_stamp': '2024-04-12T19:51:36.272283+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-12T19:51:42.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-12T19:51:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.037+0000 7f6503200640 1 Processor -- start 2024-04-12T19:51:43.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.037+0000 7f6503200640 1 -- start start 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 0x7f64fc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0cedf0 0x7f64fc0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc0a9640 con 0x7f64fc0b3e10 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc0cf250 con 0x7f64fc1461f0 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0b4be0 con 0x7f64fc0cedf0 2024-04-12T19:51:43.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6501200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6501200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53096/0 (socket says 172.21.15.32:53096) 2024-04-12T19:51:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f6501200640 1 -- 172.21.15.32:0/1722886236 learned_addr learned my addr 172.21.15.32:0/1722886236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f64f3400640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0cedf0 0x7f64fc0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.039+0000 7f64fbe00640 1 --2- 172.21.15.32:0/1722886236 >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 0x7f64fc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f6501200640 1 -- 172.21.15.32:0/1722886236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0cedf0 msgr2=0x7f64fc0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f6501200640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0cedf0 0x7f64fc0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f6501200640 1 -- 172.21.15.32:0/1722886236 >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 msgr2=0x7f64fc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f6501200640 1 --2- 172.21.15.32:0/1722886236 >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 0x7f64fc0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f6501200640 1 -- 172.21.15.32:0/1722886236 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0b4620 con 0x7f64fc0b3e10 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.040+0000 7f64fbe00640 1 --2- 172.21.15.32:0/1722886236 >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 0x7f64fc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:43.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.041+0000 7f6501200640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009e50 tx=0x7f64f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed7fc8390a6fbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.041+0000 7f64fb400640 1 -- 172.21.15.32:0/1722886236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f402ca70 con 0x7f64fc0b3e10 2024-04-12T19:51:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.041+0000 7f64fb400640 1 -- 172.21.15.32:0/1722886236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f402cc10 con 0x7f64fc0b3e10 2024-04-12T19:51:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.041+0000 7f64fb400640 1 -- 172.21.15.32:0/1722886236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40343f0 con 0x7f64fc0b3e10 2024-04-12T19:51:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 -- 172.21.15.32:0/1722886236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 msgr2=0x7f64fc0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009e50 tx=0x7f64f402bed0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 -- 172.21.15.32:0/1722886236 shutdown_connections 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc0cd2b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 --2- 172.21.15.32:0/1722886236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0cedf0 0x7f64fc0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 --2- 172.21.15.32:0/1722886236 >> v2:172.21.15.169:3300/0 conn(0x7f64fc1461f0 0x7f64fc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 -- 172.21.15.32:0/1722886236 >> 172.21.15.32:0/1722886236 conn(0x7f64fc0b27b0 msgr2=0x7f64fc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 -- 172.21.15.32:0/1722886236 shutdown_connections 2024-04-12T19:51:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.042+0000 7f6503200640 1 -- 172.21.15.32:0/1722886236 wait complete. 2024-04-12T19:51:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.044+0000 7f6503200640 1 Processor -- start 2024-04-12T19:51:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.044+0000 7f6503200640 1 -- start start 2024-04-12T19:51:43.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64fc0cedf0 0x7f64fc1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1461f0 0x7f64fc143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc15d290 con 0x7f64fc0b3e10 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc15cfb0 con 0x7f64fc0cedf0 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc1465d0 con 0x7f64fc1461f0 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53112/0 (socket says 172.21.15.32:53112) 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.045+0000 7f64fbe00640 1 -- 172.21.15.32:0/4016716310 learned_addr learned my addr 172.21.15.32:0/4016716310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64f3400640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64fc0cedf0 0x7f64fc1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f6501200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1461f0 0x7f64fc143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 -- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1461f0 msgr2=0x7f64fc143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1461f0 0x7f64fc143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 -- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64fc0cedf0 msgr2=0x7f64fc1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64fc0cedf0 0x7f64fc1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 -- 172.21.15.32:0/4016716310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc1f3d30 con 0x7f64fc0b3e10 2024-04-12T19:51:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64fbe00640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c7f0 tx=0x7f64ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec015070 con 0x7f64fc0b3e10 2024-04-12T19:51:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec007820 con 0x7f64fc0b3e10 2024-04-12T19:51:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec007bd0 con 0x7f64fc0b3e10 2024-04-12T19:51:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc1f4010 con 0x7f64fc0b3e10 2024-04-12T19:51:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.046+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc15c750 con 0x7f64fc0b3e10 2024-04-12T19:51:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.047+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc1f0ea0 con 0x7f64fc0b3e10 2024-04-12T19:51:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.049+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64ec018070 con 0x7f64fc0b3e10 2024-04-12T19:51:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.050+0000 7f64f8c00640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 0x7f64d806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.050+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f64ec08bba0 con 0x7f64fc0b3e10 2024-04-12T19:51:43.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.050+0000 7f64f3400640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 0x7f64d806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.051+0000 7f64f3400640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 0x7f64d806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0ceb90 tx=0x7f64e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.054+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64ec05a040 con 0x7f64fc0b3e10 2024-04-12T19:51:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.332+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f64fc15f8c0 con 0x7f64fc0b3e10 2024-04-12T19:51:43.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.333+0000 7f64f8c00640 1 -- 172.21.15.32:0/4016716310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1098 (secure 0 0 0) 0x7f64ec059940 con 0x7f64fc0b3e10 2024-04-12T19:51:43.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:43.333 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":105,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1712951411,"num_in_osds":8,"osd_in_since":1712951402,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":58},{"state_name":"active+clean","count":39}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":590390,"bytes_used":242475008,"bytes_avail":772851638272,"bytes_total":773094113280,"inactive_pgs_ratio":0.59793812036514282,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5022}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-12T19:51:25.646146+0000","services":{}},"progress_events":{}} 2024-04-12T19:51:43.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 msgr2=0x7f64d806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 0x7f64d806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0ceb90 tx=0x7f64e4008040 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 msgr2=0x7f64fc15c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c7f0 tx=0x7f64ec00ccb0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 shutdown_connections 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f64d806c970 0x7f64d806ee30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1461f0 0x7f64fc143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64fc0cedf0 0x7f64fc1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 --2- 172.21.15.32:0/4016716310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64fc0b3e10 0x7f64fc15c210 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 >> 172.21.15.32:0/4016716310 conn(0x7f64fc0b27b0 msgr2=0x7f64fc06e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 shutdown_connections 2024-04-12T19:51:43.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.336+0000 7f6503200640 1 -- 172.21.15.32:0/4016716310 wait complete. 2024-04-12T19:51:43.357 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-12T19:51:43.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:51:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.557+0000 7f7b2a200640 1 Processor -- start 2024-04-12T19:51:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.557+0000 7f7b2a200640 1 -- start start 2024-04-12T19:51:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b24162c80 0x7f7b240b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b240afd00 0x7f7b240ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b24071f50 0x7f7b240723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7b24072900 con 0x7f7b240afd00 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7b240b4ad0 con 0x7f7b24162c80 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b2a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b24163060 con 0x7f7b24071f50 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b240afd00 0x7f7b240ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b240afd00 0x7f7b240ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53114/0 (socket says 172.21.15.32:53114) 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 -- 172.21.15.32:0/2121399762 learned_addr learned my addr 172.21.15.32:0/2121399762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 -- 172.21.15.32:0/2121399762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b24071f50 msgr2=0x7f7b240723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 --2- 172.21.15.32:0/2121399762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b24071f50 0x7f7b240723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 -- 172.21.15.32:0/2121399762 >> v2:172.21.15.169:3300/0 conn(0x7f7b24162c80 msgr2=0x7f7b240b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 --2- 172.21.15.32:0/2121399762 >> v2:172.21.15.169:3300/0 conn(0x7f7b24162c80 0x7f7b240b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.559+0000 7f7b1be00640 1 -- 172.21.15.32:0/2121399762 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b240afae0 con 0x7f7b240afd00 2024-04-12T19:51:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.560+0000 7f7b1be00640 1 --2- 172.21.15.32:0/2121399762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b240afd00 0x7f7b240ce3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7b2406c750 tx=0x7f7b0c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ef6f34f520016f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.561+0000 7f7b23e00640 1 -- 172.21.15.32:0/2121399762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c035020 con 0x7f7b240afd00 2024-04-12T19:51:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.561+0000 7f7b23e00640 1 -- 172.21.15.32:0/2121399762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c031880 con 0x7f7b240afd00 2024-04-12T19:51:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.561+0000 7f7b23e00640 1 -- 172.21.15.32:0/2121399762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031c50 con 0x7f7b240afd00 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.561+0000 7f7b2a200640 1 -- 172.21.15.32:0/2121399762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b240afd00 msgr2=0x7f7b240ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.561+0000 7f7b2a200640 1 --2- 172.21.15.32:0/2121399762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b240afd00 0x7f7b240ce3b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7b2406c750 tx=0x7f7b0c02b920 comp rx=0 tx=0).stop 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 -- 172.21.15.32:0/2121399762 shutdown_connections 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 --2- 172.21.15.32:0/2121399762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b240afd00 0x7f7b240ce3b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 --2- 172.21.15.32:0/2121399762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b24071f50 0x7f7b240723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 --2- 172.21.15.32:0/2121399762 >> v2:172.21.15.169:3300/0 conn(0x7f7b24162c80 0x7f7b240b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 -- 172.21.15.32:0/2121399762 >> 172.21.15.32:0/2121399762 conn(0x7f7b2406b930 msgr2=0x7f7b2406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 -- 172.21.15.32:0/2121399762 shutdown_connections 2024-04-12T19:51:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.562+0000 7f7b2a200640 1 -- 172.21.15.32:0/2121399762 wait complete. 2024-04-12T19:51:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.563+0000 7f7b2a200640 1 Processor -- start 2024-04-12T19:51:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.563+0000 7f7b2a200640 1 -- start start 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b240afd00 0x7f7b241eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 0x7f7b2414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b240b3f40 con 0x7f7b24071f50 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7b240b3c60 con 0x7f7b24162c80 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.564+0000 7f7b2a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b240b4220 con 0x7f7b240afd00 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53116/0 (socket says 172.21.15.32:53116) 2024-04-12T19:51:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 -- 172.21.15.32:0/3014338705 learned_addr learned my addr 172.21.15.32:0/3014338705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 -- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b240afd00 msgr2=0x7f7b241eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b240afd00 0x7f7b241eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 -- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 msgr2=0x7f7b2414b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b29800640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 0x7f7b2414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 0x7f7b2414b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b28e00640 1 -- 172.21.15.32:0/3014338705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b24149fb0 con 0x7f7b24071f50 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.565+0000 7f7b29800640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 0x7f7b2414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b28e00640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b410 tx=0x7f7b1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14015020 con 0x7f7b24071f50 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b14011850 con 0x7f7b24071f50 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2414a290 con 0x7f7b24071f50 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14011c40 con 0x7f7b24071f50 2024-04-12T19:51:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.566+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2414a850 con 0x7f7b24071f50 2024-04-12T19:51:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.567+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b240755c0 con 0x7f7b24071f50 2024-04-12T19:51:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.568+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b14018020 con 0x7f7b24071f50 2024-04-12T19:51:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.569+0000 7f7b21600640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.569+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7b1408baf0 con 0x7f7b24071f50 2024-04-12T19:51:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.569+0000 7f7b1be00640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.570+0000 7f7b1be00640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 0x7f7b0006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036000 tx=0x7f7b0c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.574+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b1401f020 con 0x7f7b24071f50 2024-04-12T19:51:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.767+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b240b57f0 con 0x7f7b0006c8f0 2024-04-12T19:51:43.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.778+0000 7f7b21600640 1 -- 172.21.15.32:0/3014338705 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f7b240b57f0 con 0x7f7b0006c8f0 2024-04-12T19:51:43.781 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:43.783 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.783+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 msgr2=0x7f7b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.783+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 0x7f7b0006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036000 tx=0x7f7b0c030000 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 msgr2=0x7f7b241ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b410 tx=0x7f7b1400b8d0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 shutdown_connections 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7b0006c8f0 0x7f7b0006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b24162c80 0x7f7b2414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b240afd00 0x7f7b241eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 --2- 172.21.15.32:0/3014338705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b24071f50 0x7f7b241ee010 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 >> 172.21.15.32:0/3014338705 conn(0x7f7b2406b930 msgr2=0x7f7b24161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 shutdown_connections 2024-04-12T19:51:43.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:43.784+0000 7f7b2a200640 1 -- 172.21.15.32:0/3014338705 wait complete. 2024-04-12T19:51:43.800 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-12T19:51:43.650357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237092,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737628,"statfs":{"total":773094113280,"available":772851331072,"internally_reserved":0,"allocated":4366336,"data_stored":2397332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.360035"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640727+0000","last_change":"2024-04-12T19:51:37.327897+0000","last_active":"2024-04-12T19:51:39.640727+0000","last_peered":"2024-04-12T19:51:39.640727+0000","last_clean":"2024-04-12T19:51:39.640727+0000","last_became_active":"2024-04-12T19:51:37.325584+0000","last_became_peered":"2024-04-12T19:51:37.325584+0000","last_unstale":"2024-04-12T19:51:39.640727+0000","last_undegraded":"2024-04-12T19:51:39.640727+0000","last_fullsized":"2024-04-12T19:51:39.640727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327299+0000","last_change":"2024-04-12T19:51:37.313127+0000","last_active":"2024-04-12T19:51:38.327299+0000","last_peered":"2024-04-12T19:51:38.327299+0000","last_clean":"2024-04-12T19:51:38.327299+0000","last_became_active":"2024-04-12T19:51:37.312842+0000","last_became_peered":"2024-04-12T19:51:37.312842+0000","last_unstale":"2024-04-12T19:51:38.327299+0000","last_undegraded":"2024-04-12T19:51:38.327299+0000","last_fullsized":"2024-04-12T19:51:38.327299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641376+0000","last_change":"2024-04-12T19:51:37.336313+0000","last_active":"2024-04-12T19:51:39.641376+0000","last_peered":"2024-04-12T19:51:39.641376+0000","last_clean":"2024-04-12T19:51:39.641376+0000","last_became_active":"2024-04-12T19:51:37.336041+0000","last_became_peered":"2024-04-12T19:51:37.336041+0000","last_unstale":"2024-04-12T19:51:39.641376+0000","last_undegraded":"2024-04-12T19:51:39.641376+0000","last_fullsized":"2024-04-12T19:51:39.641376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:45.554666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540672+0000","last_change":"2024-04-12T19:51:37.342853+0000","last_active":"2024-04-12T19:51:38.540672+0000","last_peered":"2024-04-12T19:51:38.540672+0000","last_clean":"2024-04-12T19:51:38.540672+0000","last_became_active":"2024-04-12T19:51:37.342521+0000","last_became_peered":"2024-04-12T19:51:37.342521+0000","last_unstale":"2024-04-12T19:51:38.540672+0000","last_undegraded":"2024-04-12T19:51:38.540672+0000","last_fullsized":"2024-04-12T19:51:38.540672+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:30:46.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326477+0000","last_change":"2024-04-12T19:51:37.324560+0000","last_active":"2024-04-12T19:51:38.326477+0000","last_peered":"2024-04-12T19:51:38.326477+0000","last_clean":"2024-04-12T19:51:38.326477+0000","last_became_active":"2024-04-12T19:51:37.324249+0000","last_became_peered":"2024-04-12T19:51:37.324249+0000","last_unstale":"2024-04-12T19:51:38.326477+0000","last_undegraded":"2024-04-12T19:51:38.326477+0000","last_fullsized":"2024-04-12T19:51:38.326477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327988+0000","last_change":"2024-04-12T19:51:37.335976+0000","last_active":"2024-04-12T19:51:38.327988+0000","last_peered":"2024-04-12T19:51:38.327988+0000","last_clean":"2024-04-12T19:51:38.327988+0000","last_became_active":"2024-04-12T19:51:37.335672+0000","last_became_peered":"2024-04-12T19:51:37.335672+0000","last_unstale":"2024-04-12T19:51:38.327988+0000","last_undegraded":"2024-04-12T19:51:38.327988+0000","last_fullsized":"2024-04-12T19:51:38.327988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:11:45.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642271+0000","last_change":"2024-04-12T19:51:37.326877+0000","last_active":"2024-04-12T19:51:39.642271+0000","last_peered":"2024-04-12T19:51:39.642271+0000","last_clean":"2024-04-12T19:51:39.642271+0000","last_became_active":"2024-04-12T19:51:37.326158+0000","last_became_peered":"2024-04-12T19:51:37.326158+0000","last_unstale":"2024-04-12T19:51:39.642271+0000","last_undegraded":"2024-04-12T19:51:39.642271+0000","last_fullsized":"2024-04-12T19:51:39.642271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:30.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642350+0000","last_change":"2024-04-12T19:51:37.335344+0000","last_active":"2024-04-12T19:51:39.642350+0000","last_peered":"2024-04-12T19:51:39.642350+0000","last_clean":"2024-04-12T19:51:39.642350+0000","last_became_active":"2024-04-12T19:51:37.328030+0000","last_became_peered":"2024-04-12T19:51:37.328030+0000","last_unstale":"2024-04-12T19:51:39.642350+0000","last_undegraded":"2024-04-12T19:51:39.642350+0000","last_fullsized":"2024-04-12T19:51:39.642350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:46.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642013+0000","last_change":"2024-04-12T19:51:37.315564+0000","last_active":"2024-04-12T19:51:39.642013+0000","last_peered":"2024-04-12T19:51:39.642013+0000","last_clean":"2024-04-12T19:51:39.642013+0000","last_became_active":"2024-04-12T19:51:37.315178+0000","last_became_peered":"2024-04-12T19:51:37.315178+0000","last_unstale":"2024-04-12T19:51:39.642013+0000","last_undegraded":"2024-04-12T19:51:39.642013+0000","last_fullsized":"2024-04-12T19:51:39.642013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:57.161489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541316+0000","last_change":"2024-04-12T19:51:37.301078+0000","last_active":"2024-04-12T19:51:38.541316+0000","last_peered":"2024-04-12T19:51:38.541316+0000","last_clean":"2024-04-12T19:51:38.541316+0000","last_became_active":"2024-04-12T19:51:37.300553+0000","last_became_peered":"2024-04-12T19:51:37.300553+0000","last_unstale":"2024-04-12T19:51:38.541316+0000","last_undegraded":"2024-04-12T19:51:38.541316+0000","last_fullsized":"2024-04-12T19:51:38.541316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:39:03.668325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322081+0000","last_change":"2024-04-12T19:51:37.313139+0000","last_active":"2024-04-12T19:51:38.322081+0000","last_peered":"2024-04-12T19:51:38.322081+0000","last_clean":"2024-04-12T19:51:38.322081+0000","last_became_active":"2024-04-12T19:51:37.312638+0000","last_became_peered":"2024-04-12T19:51:37.312638+0000","last_unstale":"2024-04-12T19:51:38.322081+0000","last_undegraded":"2024-04-12T19:51:38.322081+0000","last_fullsized":"2024-04-12T19:51:38.322081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:45.128642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640767+0000","last_change":"2024-04-12T19:51:37.320540+0000","last_active":"2024-04-12T19:51:39.640767+0000","last_peered":"2024-04-12T19:51:39.640767+0000","last_clean":"2024-04-12T19:51:39.640767+0000","last_became_active":"2024-04-12T19:51:37.320205+0000","last_became_peered":"2024-04-12T19:51:37.320205+0000","last_unstale":"2024-04-12T19:51:39.640767+0000","last_undegraded":"2024-04-12T19:51:39.640767+0000","last_fullsized":"2024-04-12T19:51:39.640767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:38.119800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598554+0000","last_change":"2024-04-12T19:51:37.322595+0000","last_active":"2024-04-12T19:51:39.598554+0000","last_peered":"2024-04-12T19:51:39.598554+0000","last_clean":"2024-04-12T19:51:39.598554+0000","last_became_active":"2024-04-12T19:51:37.321210+0000","last_became_peered":"2024-04-12T19:51:37.321210+0000","last_unstale":"2024-04-12T19:51:39.598554+0000","last_undegraded":"2024-04-12T19:51:39.598554+0000","last_fullsized":"2024-04-12T19:51:39.598554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540450+0000","last_change":"2024-04-12T19:51:37.301269+0000","last_active":"2024-04-12T19:51:38.540450+0000","last_peered":"2024-04-12T19:51:38.540450+0000","last_clean":"2024-04-12T19:51:38.540450+0000","last_became_active":"2024-04-12T19:51:37.300540+0000","last_became_peered":"2024-04-12T19:51:37.300540+0000","last_unstale":"2024-04-12T19:51:38.540450+0000","last_undegraded":"2024-04-12T19:51:38.540450+0000","last_fullsized":"2024-04-12T19:51:38.540450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:19:32.067053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607685+0000","last_change":"2024-04-12T19:51:37.329994+0000","last_active":"2024-04-12T19:51:39.607685+0000","last_peered":"2024-04-12T19:51:39.607685+0000","last_clean":"2024-04-12T19:51:39.607685+0000","last_became_active":"2024-04-12T19:51:37.329685+0000","last_became_peered":"2024-04-12T19:51:37.329685+0000","last_unstale":"2024-04-12T19:51:39.607685+0000","last_undegraded":"2024-04-12T19:51:39.607685+0000","last_fullsized":"2024-04-12T19:51:39.607685+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607696+0000","last_change":"2024-04-12T19:51:37.318607+0000","last_active":"2024-04-12T19:51:39.607696+0000","last_peered":"2024-04-12T19:51:39.607696+0000","last_clean":"2024-04-12T19:51:39.607696+0000","last_became_active":"2024-04-12T19:51:37.318119+0000","last_became_peered":"2024-04-12T19:51:37.318119+0000","last_unstale":"2024-04-12T19:51:39.607696+0000","last_undegraded":"2024-04-12T19:51:39.607696+0000","last_fullsized":"2024-04-12T19:51:39.607696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321833+0000","last_change":"2024-04-12T19:51:37.322110+0000","last_active":"2024-04-12T19:51:38.321833+0000","last_peered":"2024-04-12T19:51:38.321833+0000","last_clean":"2024-04-12T19:51:38.321833+0000","last_became_active":"2024-04-12T19:51:37.321785+0000","last_became_peered":"2024-04-12T19:51:37.321785+0000","last_unstale":"2024-04-12T19:51:38.321833+0000","last_undegraded":"2024-04-12T19:51:38.321833+0000","last_fullsized":"2024-04-12T19:51:38.321833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:23.809767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327881+0000","last_change":"2024-04-12T19:51:37.326355+0000","last_active":"2024-04-12T19:51:38.327881+0000","last_peered":"2024-04-12T19:51:38.327881+0000","last_clean":"2024-04-12T19:51:38.327881+0000","last_became_active":"2024-04-12T19:51:37.326026+0000","last_became_peered":"2024-04-12T19:51:37.326026+0000","last_unstale":"2024-04-12T19:51:38.327881+0000","last_undegraded":"2024-04-12T19:51:38.327881+0000","last_fullsized":"2024-04-12T19:51:38.327881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:15:16.329928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328156+0000","last_change":"2024-04-12T19:51:37.328370+0000","last_active":"2024-04-12T19:51:38.328156+0000","last_peered":"2024-04-12T19:51:38.328156+0000","last_clean":"2024-04-12T19:51:38.328156+0000","last_became_active":"2024-04-12T19:51:37.328016+0000","last_became_peered":"2024-04-12T19:51:37.328016+0000","last_unstale":"2024-04-12T19:51:38.328156+0000","last_undegraded":"2024-04-12T19:51:38.328156+0000","last_fullsized":"2024-04-12T19:51:38.328156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:59:33.416853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598843+0000","last_change":"2024-04-12T19:51:37.319605+0000","last_active":"2024-04-12T19:51:39.598843+0000","last_peered":"2024-04-12T19:51:39.598843+0000","last_clean":"2024-04-12T19:51:39.598843+0000","last_became_active":"2024-04-12T19:51:37.319245+0000","last_became_peered":"2024-04-12T19:51:37.319245+0000","last_unstale":"2024-04-12T19:51:39.598843+0000","last_undegraded":"2024-04-12T19:51:39.598843+0000","last_fullsized":"2024-04-12T19:51:39.598843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607827+0000","last_change":"2024-04-12T19:51:37.320857+0000","last_active":"2024-04-12T19:51:39.607827+0000","last_peered":"2024-04-12T19:51:39.607827+0000","last_clean":"2024-04-12T19:51:39.607827+0000","last_became_active":"2024-04-12T19:51:37.320526+0000","last_became_peered":"2024-04-12T19:51:37.320526+0000","last_unstale":"2024-04-12T19:51:39.607827+0000","last_undegraded":"2024-04-12T19:51:39.607827+0000","last_fullsized":"2024-04-12T19:51:39.607827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641608+0000","last_change":"2024-04-12T19:51:37.325464+0000","last_active":"2024-04-12T19:51:39.641608+0000","last_peered":"2024-04-12T19:51:39.641608+0000","last_clean":"2024-04-12T19:51:39.641608+0000","last_became_active":"2024-04-12T19:51:37.325178+0000","last_became_peered":"2024-04-12T19:51:37.325178+0000","last_unstale":"2024-04-12T19:51:39.641608+0000","last_undegraded":"2024-04-12T19:51:39.641608+0000","last_fullsized":"2024-04-12T19:51:39.641608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598096+0000","last_change":"2024-04-12T19:51:37.333175+0000","last_active":"2024-04-12T19:51:39.598096+0000","last_peered":"2024-04-12T19:51:39.598096+0000","last_clean":"2024-04-12T19:51:39.598096+0000","last_became_active":"2024-04-12T19:51:37.332867+0000","last_became_peered":"2024-04-12T19:51:37.332867+0000","last_unstale":"2024-04-12T19:51:39.598096+0000","last_undegraded":"2024-04-12T19:51:39.598096+0000","last_fullsized":"2024-04-12T19:51:39.598096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:13.204204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642594+0000","last_change":"2024-04-12T19:51:37.328365+0000","last_active":"2024-04-12T19:51:39.642594+0000","last_peered":"2024-04-12T19:51:39.642594+0000","last_clean":"2024-04-12T19:51:39.642594+0000","last_became_active":"2024-04-12T19:51:37.327327+0000","last_became_peered":"2024-04-12T19:51:37.327327+0000","last_unstale":"2024-04-12T19:51:39.642594+0000","last_undegraded":"2024-04-12T19:51:39.642594+0000","last_fullsized":"2024-04-12T19:51:39.642594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:30:46.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541461+0000","last_change":"2024-04-12T19:51:37.312148+0000","last_active":"2024-04-12T19:51:38.541461+0000","last_peered":"2024-04-12T19:51:38.541461+0000","last_clean":"2024-04-12T19:51:38.541461+0000","last_became_active":"2024-04-12T19:51:37.311795+0000","last_became_peered":"2024-04-12T19:51:37.311795+0000","last_unstale":"2024-04-12T19:51:38.541461+0000","last_undegraded":"2024-04-12T19:51:38.541461+0000","last_fullsized":"2024-04-12T19:51:38.541461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:44.292363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541572+0000","last_change":"2024-04-12T19:51:37.316522+0000","last_active":"2024-04-12T19:51:38.541572+0000","last_peered":"2024-04-12T19:51:38.541572+0000","last_clean":"2024-04-12T19:51:38.541572+0000","last_became_active":"2024-04-12T19:51:37.316208+0000","last_became_peered":"2024-04-12T19:51:37.316208+0000","last_unstale":"2024-04-12T19:51:38.541572+0000","last_undegraded":"2024-04-12T19:51:38.541572+0000","last_fullsized":"2024-04-12T19:51:38.541572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:49:38.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326806+0000","last_change":"2024-04-12T19:51:37.308586+0000","last_active":"2024-04-12T19:51:38.326806+0000","last_peered":"2024-04-12T19:51:38.326806+0000","last_clean":"2024-04-12T19:51:38.326806+0000","last_became_active":"2024-04-12T19:51:37.306339+0000","last_became_peered":"2024-04-12T19:51:37.306339+0000","last_unstale":"2024-04-12T19:51:38.326806+0000","last_undegraded":"2024-04-12T19:51:38.326806+0000","last_fullsized":"2024-04-12T19:51:38.326806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640473+0000","last_change":"2024-04-12T19:51:37.337100+0000","last_active":"2024-04-12T19:51:39.640473+0000","last_peered":"2024-04-12T19:51:39.640473+0000","last_clean":"2024-04-12T19:51:39.640473+0000","last_became_active":"2024-04-12T19:51:37.336677+0000","last_became_peered":"2024-04-12T19:51:37.336677+0000","last_unstale":"2024-04-12T19:51:39.640473+0000","last_undegraded":"2024-04-12T19:51:39.640473+0000","last_fullsized":"2024-04-12T19:51:39.640473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608098+0000","last_change":"2024-04-12T19:51:37.301241+0000","last_active":"2024-04-12T19:51:39.608098+0000","last_peered":"2024-04-12T19:51:39.608098+0000","last_clean":"2024-04-12T19:51:39.608098+0000","last_became_active":"2024-04-12T19:51:37.300515+0000","last_became_peered":"2024-04-12T19:51:37.300515+0000","last_unstale":"2024-04-12T19:51:39.608098+0000","last_undegraded":"2024-04-12T19:51:39.608098+0000","last_fullsized":"2024-04-12T19:51:39.608098+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607767+0000","last_change":"2024-04-12T19:51:37.319225+0000","last_active":"2024-04-12T19:51:39.607767+0000","last_peered":"2024-04-12T19:51:39.607767+0000","last_clean":"2024-04-12T19:51:39.607767+0000","last_became_active":"2024-04-12T19:51:37.317746+0000","last_became_peered":"2024-04-12T19:51:37.317746+0000","last_unstale":"2024-04-12T19:51:39.607767+0000","last_undegraded":"2024-04-12T19:51:39.607767+0000","last_fullsized":"2024-04-12T19:51:39.607767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641623+0000","last_change":"2024-04-12T19:51:37.327074+0000","last_active":"2024-04-12T19:51:39.641623+0000","last_peered":"2024-04-12T19:51:39.641623+0000","last_clean":"2024-04-12T19:51:39.641623+0000","last_became_active":"2024-04-12T19:51:37.326493+0000","last_became_peered":"2024-04-12T19:51:37.326493+0000","last_unstale":"2024-04-12T19:51:39.641623+0000","last_undegraded":"2024-04-12T19:51:39.641623+0000","last_fullsized":"2024-04-12T19:51:39.641623+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321339+0000","last_change":"2024-04-12T19:51:37.332286+0000","last_active":"2024-04-12T19:51:38.321339+0000","last_peered":"2024-04-12T19:51:38.321339+0000","last_clean":"2024-04-12T19:51:38.321339+0000","last_became_active":"2024-04-12T19:51:37.330122+0000","last_became_peered":"2024-04-12T19:51:37.330122+0000","last_unstale":"2024-04-12T19:51:38.321339+0000","last_undegraded":"2024-04-12T19:51:38.321339+0000","last_fullsized":"2024-04-12T19:51:38.321339+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:20:56.060888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.599010+0000","last_change":"2024-04-12T19:51:37.322199+0000","last_active":"2024-04-12T19:51:39.599010+0000","last_peered":"2024-04-12T19:51:39.599010+0000","last_clean":"2024-04-12T19:51:39.599010+0000","last_became_active":"2024-04-12T19:51:37.321849+0000","last_became_peered":"2024-04-12T19:51:37.321849+0000","last_unstale":"2024-04-12T19:51:39.599010+0000","last_undegraded":"2024-04-12T19:51:39.599010+0000","last_fullsized":"2024-04-12T19:51:39.599010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322105+0000","last_change":"2024-04-12T19:51:36.323028+0000","last_active":"2024-04-12T19:51:38.322105+0000","last_peered":"2024-04-12T19:51:38.322105+0000","last_clean":"2024-04-12T19:51:38.322105+0000","last_became_active":"2024-04-12T19:51:36.322694+0000","last_became_peered":"2024-04-12T19:51:36.322694+0000","last_unstale":"2024-04-12T19:51:38.322105+0000","last_undegraded":"2024-04-12T19:51:38.322105+0000","last_fullsized":"2024-04-12T19:51:38.322105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:29.889791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641057+0000","last_change":"2024-04-12T19:51:37.334931+0000","last_active":"2024-04-12T19:51:39.641057+0000","last_peered":"2024-04-12T19:51:39.641057+0000","last_clean":"2024-04-12T19:51:39.641057+0000","last_became_active":"2024-04-12T19:51:37.331107+0000","last_became_peered":"2024-04-12T19:51:37.331107+0000","last_unstale":"2024-04-12T19:51:39.641057+0000","last_undegraded":"2024-04-12T19:51:39.641057+0000","last_fullsized":"2024-04-12T19:51:39.641057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541600+0000","last_change":"2024-04-12T19:51:36.325298+0000","last_active":"2024-04-12T19:51:38.541600+0000","last_peered":"2024-04-12T19:51:38.541600+0000","last_clean":"2024-04-12T19:51:38.541600+0000","last_became_active":"2024-04-12T19:51:36.324222+0000","last_became_peered":"2024-04-12T19:51:36.324222+0000","last_unstale":"2024-04-12T19:51:38.541600+0000","last_undegraded":"2024-04-12T19:51:38.541600+0000","last_fullsized":"2024-04-12T19:51:38.541600+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:44:36.049413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640563+0000","last_change":"2024-04-12T19:51:37.322294+0000","last_active":"2024-04-12T19:51:39.640563+0000","last_peered":"2024-04-12T19:51:39.640563+0000","last_clean":"2024-04-12T19:51:39.640563+0000","last_became_active":"2024-04-12T19:51:37.321974+0000","last_became_peered":"2024-04-12T19:51:37.321974+0000","last_unstale":"2024-04-12T19:51:39.640563+0000","last_undegraded":"2024-04-12T19:51:39.640563+0000","last_fullsized":"2024-04-12T19:51:39.640563+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328355+0000","last_change":"2024-04-12T19:51:36.331707+0000","last_active":"2024-04-12T19:51:38.328355+0000","last_peered":"2024-04-12T19:51:38.328355+0000","last_clean":"2024-04-12T19:51:38.328355+0000","last_became_active":"2024-04-12T19:51:36.331198+0000","last_became_peered":"2024-04-12T19:51:36.331198+0000","last_unstale":"2024-04-12T19:51:38.328355+0000","last_undegraded":"2024-04-12T19:51:38.328355+0000","last_fullsized":"2024-04-12T19:51:38.328355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:18.552643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540967+0000","last_change":"2024-04-12T19:51:37.325365+0000","last_active":"2024-04-12T19:51:38.540967+0000","last_peered":"2024-04-12T19:51:38.540967+0000","last_clean":"2024-04-12T19:51:38.540967+0000","last_became_active":"2024-04-12T19:51:37.323723+0000","last_became_peered":"2024-04-12T19:51:37.323723+0000","last_unstale":"2024-04-12T19:51:38.540967+0000","last_undegraded":"2024-04-12T19:51:38.540967+0000","last_fullsized":"2024-04-12T19:51:38.540967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:49.343920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541024+0000","last_change":"2024-04-12T19:51:36.330056+0000","last_active":"2024-04-12T19:51:38.541024+0000","last_peered":"2024-04-12T19:51:38.541024+0000","last_clean":"2024-04-12T19:51:38.541024+0000","last_became_active":"2024-04-12T19:51:36.329657+0000","last_became_peered":"2024-04-12T19:51:36.329657+0000","last_unstale":"2024-04-12T19:51:38.541024+0000","last_undegraded":"2024-04-12T19:51:38.541024+0000","last_fullsized":"2024-04-12T19:51:38.541024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640592+0000","last_change":"2024-04-12T19:51:37.317712+0000","last_active":"2024-04-12T19:51:39.640592+0000","last_peered":"2024-04-12T19:51:39.640592+0000","last_clean":"2024-04-12T19:51:39.640592+0000","last_became_active":"2024-04-12T19:51:37.317351+0000","last_became_peered":"2024-04-12T19:51:37.317351+0000","last_unstale":"2024-04-12T19:51:39.640592+0000","last_undegraded":"2024-04-12T19:51:39.640592+0000","last_fullsized":"2024-04-12T19:51:39.640592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:42.048987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640936+0000","last_change":"2024-04-12T19:51:36.327409+0000","last_active":"2024-04-12T19:51:39.640936+0000","last_peered":"2024-04-12T19:51:39.640936+0000","last_clean":"2024-04-12T19:51:39.640936+0000","last_became_active":"2024-04-12T19:51:36.327107+0000","last_became_peered":"2024-04-12T19:51:36.327107+0000","last_unstale":"2024-04-12T19:51:39.640936+0000","last_undegraded":"2024-04-12T19:51:39.640936+0000","last_fullsized":"2024-04-12T19:51:39.640936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:40:21.315726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640739+0000","last_change":"2024-04-12T19:51:37.327878+0000","last_active":"2024-04-12T19:51:39.640739+0000","last_peered":"2024-04-12T19:51:39.640739+0000","last_clean":"2024-04-12T19:51:39.640739+0000","last_became_active":"2024-04-12T19:51:37.327592+0000","last_became_peered":"2024-04-12T19:51:37.327592+0000","last_unstale":"2024-04-12T19:51:39.640739+0000","last_undegraded":"2024-04-12T19:51:39.640739+0000","last_fullsized":"2024-04-12T19:51:39.640739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641467+0000","last_change":"2024-04-12T19:51:36.324064+0000","last_active":"2024-04-12T19:51:39.641467+0000","last_peered":"2024-04-12T19:51:39.641467+0000","last_clean":"2024-04-12T19:51:39.641467+0000","last_became_active":"2024-04-12T19:51:36.323334+0000","last_became_peered":"2024-04-12T19:51:36.323334+0000","last_unstale":"2024-04-12T19:51:39.641467+0000","last_undegraded":"2024-04-12T19:51:39.641467+0000","last_fullsized":"2024-04-12T19:51:39.641467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640556+0000","last_change":"2024-04-12T19:51:37.322281+0000","last_active":"2024-04-12T19:51:39.640556+0000","last_peered":"2024-04-12T19:51:39.640556+0000","last_clean":"2024-04-12T19:51:39.640556+0000","last_became_active":"2024-04-12T19:51:37.318930+0000","last_became_peered":"2024-04-12T19:51:37.318930+0000","last_unstale":"2024-04-12T19:51:39.640556+0000","last_undegraded":"2024-04-12T19:51:39.640556+0000","last_fullsized":"2024-04-12T19:51:39.640556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:48.497592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326879+0000","last_change":"2024-04-12T19:51:36.326080+0000","last_active":"2024-04-12T19:51:38.326879+0000","last_peered":"2024-04-12T19:51:38.326879+0000","last_clean":"2024-04-12T19:51:38.326879+0000","last_became_active":"2024-04-12T19:51:36.325817+0000","last_became_peered":"2024-04-12T19:51:36.325817+0000","last_unstale":"2024-04-12T19:51:38.326879+0000","last_undegraded":"2024-04-12T19:51:38.326879+0000","last_fullsized":"2024-04-12T19:51:38.326879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:03.515939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.539962+0000","last_change":"2024-04-12T19:51:37.312621+0000","last_active":"2024-04-12T19:51:38.539962+0000","last_peered":"2024-04-12T19:51:38.539962+0000","last_clean":"2024-04-12T19:51:38.539962+0000","last_became_active":"2024-04-12T19:51:37.312092+0000","last_became_peered":"2024-04-12T19:51:37.312092+0000","last_unstale":"2024-04-12T19:51:38.539962+0000","last_undegraded":"2024-04-12T19:51:38.539962+0000","last_fullsized":"2024-04-12T19:51:38.539962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642833+0000","last_change":"2024-04-12T19:51:36.320855+0000","last_active":"2024-04-12T19:51:39.642833+0000","last_peered":"2024-04-12T19:51:39.642833+0000","last_clean":"2024-04-12T19:51:39.642833+0000","last_became_active":"2024-04-12T19:51:36.320566+0000","last_became_peered":"2024-04-12T19:51:36.320566+0000","last_unstale":"2024-04-12T19:51:39.642833+0000","last_undegraded":"2024-04-12T19:51:39.642833+0000","last_fullsized":"2024-04-12T19:51:39.642833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321976+0000","last_change":"2024-04-12T19:51:37.328807+0000","last_active":"2024-04-12T19:51:38.321976+0000","last_peered":"2024-04-12T19:51:38.321976+0000","last_clean":"2024-04-12T19:51:38.321976+0000","last_became_active":"2024-04-12T19:51:37.323818+0000","last_became_peered":"2024-04-12T19:51:37.323818+0000","last_unstale":"2024-04-12T19:51:38.321976+0000","last_undegraded":"2024-04-12T19:51:38.321976+0000","last_fullsized":"2024-04-12T19:51:38.321976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:48:11.338822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327684+0000","last_change":"2024-04-12T19:51:36.323202+0000","last_active":"2024-04-12T19:51:38.327684+0000","last_peered":"2024-04-12T19:51:38.327684+0000","last_clean":"2024-04-12T19:51:38.327684+0000","last_became_active":"2024-04-12T19:51:36.322883+0000","last_became_peered":"2024-04-12T19:51:36.322883+0000","last_unstale":"2024-04-12T19:51:38.327684+0000","last_undegraded":"2024-04-12T19:51:38.327684+0000","last_fullsized":"2024-04-12T19:51:38.327684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:42:56.574639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541543+0000","last_change":"2024-04-12T19:51:37.309643+0000","last_active":"2024-04-12T19:51:38.541543+0000","last_peered":"2024-04-12T19:51:38.541543+0000","last_clean":"2024-04-12T19:51:38.541543+0000","last_became_active":"2024-04-12T19:51:37.303858+0000","last_became_peered":"2024-04-12T19:51:37.303858+0000","last_unstale":"2024-04-12T19:51:38.541543+0000","last_undegraded":"2024-04-12T19:51:38.541543+0000","last_fullsized":"2024-04-12T19:51:38.541543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:03:09.445009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327103+0000","last_change":"2024-04-12T19:51:36.327948+0000","last_active":"2024-04-12T19:51:38.327103+0000","last_peered":"2024-04-12T19:51:38.327103+0000","last_clean":"2024-04-12T19:51:38.327103+0000","last_became_active":"2024-04-12T19:51:36.327654+0000","last_became_peered":"2024-04-12T19:51:36.327654+0000","last_unstale":"2024-04-12T19:51:38.327103+0000","last_undegraded":"2024-04-12T19:51:38.327103+0000","last_fullsized":"2024-04-12T19:51:38.327103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:17:47.877823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540673+0000","last_change":"2024-04-12T19:51:37.315215+0000","last_active":"2024-04-12T19:51:38.540673+0000","last_peered":"2024-04-12T19:51:38.540673+0000","last_clean":"2024-04-12T19:51:38.540673+0000","last_became_active":"2024-04-12T19:51:37.314923+0000","last_became_peered":"2024-04-12T19:51:37.314923+0000","last_unstale":"2024-04-12T19:51:38.540673+0000","last_undegraded":"2024-04-12T19:51:38.540673+0000","last_fullsized":"2024-04-12T19:51:38.540673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:46.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321773+0000","last_change":"2024-04-12T19:51:36.304808+0000","last_active":"2024-04-12T19:51:38.321773+0000","last_peered":"2024-04-12T19:51:38.321773+0000","last_clean":"2024-04-12T19:51:38.321773+0000","last_became_active":"2024-04-12T19:51:36.304470+0000","last_became_peered":"2024-04-12T19:51:36.304470+0000","last_unstale":"2024-04-12T19:51:38.321773+0000","last_undegraded":"2024-04-12T19:51:38.321773+0000","last_fullsized":"2024-04-12T19:51:38.321773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:24.374284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321768+0000","last_change":"2024-04-12T19:51:37.312919+0000","last_active":"2024-04-12T19:51:38.321768+0000","last_peered":"2024-04-12T19:51:38.321768+0000","last_clean":"2024-04-12T19:51:38.321768+0000","last_became_active":"2024-04-12T19:51:37.312195+0000","last_became_peered":"2024-04-12T19:51:37.312195+0000","last_unstale":"2024-04-12T19:51:38.321768+0000","last_undegraded":"2024-04-12T19:51:38.321768+0000","last_fullsized":"2024-04-12T19:51:38.321768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:03:31.127824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598741+0000","last_change":"2024-04-12T19:51:36.331502+0000","last_active":"2024-04-12T19:51:39.598741+0000","last_peered":"2024-04-12T19:51:39.598741+0000","last_clean":"2024-04-12T19:51:39.598741+0000","last_became_active":"2024-04-12T19:51:36.329297+0000","last_became_peered":"2024-04-12T19:51:36.329297+0000","last_unstale":"2024-04-12T19:51:39.598741+0000","last_undegraded":"2024-04-12T19:51:39.598741+0000","last_fullsized":"2024-04-12T19:51:39.598741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598033+0000","last_change":"2024-04-12T19:51:37.320432+0000","last_active":"2024-04-12T19:51:39.598033+0000","last_peered":"2024-04-12T19:51:39.598033+0000","last_clean":"2024-04-12T19:51:39.598033+0000","last_became_active":"2024-04-12T19:51:37.319862+0000","last_became_peered":"2024-04-12T19:51:37.319862+0000","last_unstale":"2024-04-12T19:51:39.598033+0000","last_undegraded":"2024-04-12T19:51:39.598033+0000","last_fullsized":"2024-04-12T19:51:39.598033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:22:08.107072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641138+0000","last_change":"2024-04-12T19:51:36.316118+0000","last_active":"2024-04-12T19:51:39.641138+0000","last_peered":"2024-04-12T19:51:39.641138+0000","last_clean":"2024-04-12T19:51:39.641138+0000","last_became_active":"2024-04-12T19:51:36.315852+0000","last_became_peered":"2024-04-12T19:51:36.315852+0000","last_unstale":"2024-04-12T19:51:39.641138+0000","last_undegraded":"2024-04-12T19:51:39.641138+0000","last_fullsized":"2024-04-12T19:51:39.641138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541341+0000","last_change":"2024-04-12T19:51:37.325180+0000","last_active":"2024-04-12T19:51:38.541341+0000","last_peered":"2024-04-12T19:51:38.541341+0000","last_clean":"2024-04-12T19:51:38.541341+0000","last_became_active":"2024-04-12T19:51:37.323566+0000","last_became_peered":"2024-04-12T19:51:37.323566+0000","last_unstale":"2024-04-12T19:51:38.541341+0000","last_undegraded":"2024-04-12T19:51:38.541341+0000","last_fullsized":"2024-04-12T19:51:38.541341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:58:36.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540061+0000","last_change":"2024-04-12T19:51:36.320973+0000","last_active":"2024-04-12T19:51:38.540061+0000","last_peered":"2024-04-12T19:51:38.540061+0000","last_clean":"2024-04-12T19:51:38.540061+0000","last_became_active":"2024-04-12T19:51:36.320688+0000","last_became_peered":"2024-04-12T19:51:36.320688+0000","last_unstale":"2024-04-12T19:51:38.540061+0000","last_undegraded":"2024-04-12T19:51:38.540061+0000","last_fullsized":"2024-04-12T19:51:38.540061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642731+0000","last_change":"2024-04-12T19:51:37.327683+0000","last_active":"2024-04-12T19:51:39.642731+0000","last_peered":"2024-04-12T19:51:39.642731+0000","last_clean":"2024-04-12T19:51:39.642731+0000","last_became_active":"2024-04-12T19:51:37.326936+0000","last_became_peered":"2024-04-12T19:51:37.326936+0000","last_unstale":"2024-04-12T19:51:39.642731+0000","last_undegraded":"2024-04-12T19:51:39.642731+0000","last_fullsized":"2024-04-12T19:51:39.642731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:40:58.067392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.539745+0000","last_change":"2024-04-12T19:51:36.313661+0000","last_active":"2024-04-12T19:51:38.539745+0000","last_peered":"2024-04-12T19:51:38.539745+0000","last_clean":"2024-04-12T19:51:38.539745+0000","last_became_active":"2024-04-12T19:51:36.313249+0000","last_became_peered":"2024-04-12T19:51:36.313249+0000","last_unstale":"2024-04-12T19:51:38.539745+0000","last_undegraded":"2024-04-12T19:51:38.539745+0000","last_fullsized":"2024-04-12T19:51:38.539745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321330+0000","last_change":"2024-04-12T19:51:37.330768+0000","last_active":"2024-04-12T19:51:38.321330+0000","last_peered":"2024-04-12T19:51:38.321330+0000","last_clean":"2024-04-12T19:51:38.321330+0000","last_became_active":"2024-04-12T19:51:37.329678+0000","last_became_peered":"2024-04-12T19:51:37.329678+0000","last_unstale":"2024-04-12T19:51:38.321330+0000","last_undegraded":"2024-04-12T19:51:38.321330+0000","last_fullsized":"2024-04-12T19:51:38.321330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:10:30.334195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326281+0000","last_change":"2024-04-12T19:51:36.324874+0000","last_active":"2024-04-12T19:51:38.326281+0000","last_peered":"2024-04-12T19:51:38.326281+0000","last_clean":"2024-04-12T19:51:38.326281+0000","last_became_active":"2024-04-12T19:51:36.324013+0000","last_became_peered":"2024-04-12T19:51:36.324013+0000","last_unstale":"2024-04-12T19:51:38.326281+0000","last_undegraded":"2024-04-12T19:51:38.326281+0000","last_fullsized":"2024-04-12T19:51:38.326281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:10:23.725348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322274+0000","last_change":"2024-04-12T19:51:37.319644+0000","last_active":"2024-04-12T19:51:38.322274+0000","last_peered":"2024-04-12T19:51:38.322274+0000","last_clean":"2024-04-12T19:51:38.322274+0000","last_became_active":"2024-04-12T19:51:37.319356+0000","last_became_peered":"2024-04-12T19:51:37.319356+0000","last_unstale":"2024-04-12T19:51:38.322274+0000","last_undegraded":"2024-04-12T19:51:38.322274+0000","last_fullsized":"2024-04-12T19:51:38.322274+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:14:26.033574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328103+0000","last_change":"2024-04-12T19:51:36.332183+0000","last_active":"2024-04-12T19:51:38.328103+0000","last_peered":"2024-04-12T19:51:38.328103+0000","last_clean":"2024-04-12T19:51:38.328103+0000","last_became_active":"2024-04-12T19:51:36.331021+0000","last_became_peered":"2024-04-12T19:51:36.331021+0000","last_unstale":"2024-04-12T19:51:38.328103+0000","last_undegraded":"2024-04-12T19:51:38.328103+0000","last_fullsized":"2024-04-12T19:51:38.328103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:29:38.531795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642261+0000","last_change":"2024-04-12T19:51:37.314878+0000","last_active":"2024-04-12T19:51:39.642261+0000","last_peered":"2024-04-12T19:51:39.642261+0000","last_clean":"2024-04-12T19:51:39.642261+0000","last_became_active":"2024-04-12T19:51:37.314541+0000","last_became_peered":"2024-04-12T19:51:37.314541+0000","last_unstale":"2024-04-12T19:51:39.642261+0000","last_undegraded":"2024-04-12T19:51:39.642261+0000","last_fullsized":"2024-04-12T19:51:39.642261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:54.323339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640369+0000","last_change":"2024-04-12T19:51:36.311352+0000","last_active":"2024-04-12T19:51:39.640369+0000","last_peered":"2024-04-12T19:51:39.640369+0000","last_clean":"2024-04-12T19:51:39.640369+0000","last_became_active":"2024-04-12T19:51:36.310541+0000","last_became_peered":"2024-04-12T19:51:36.310541+0000","last_unstale":"2024-04-12T19:51:39.640369+0000","last_undegraded":"2024-04-12T19:51:39.640369+0000","last_fullsized":"2024-04-12T19:51:39.640369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326234+0000","last_change":"2024-04-12T19:51:37.317144+0000","last_active":"2024-04-12T19:51:38.326234+0000","last_peered":"2024-04-12T19:51:38.326234+0000","last_clean":"2024-04-12T19:51:38.326234+0000","last_became_active":"2024-04-12T19:51:37.316891+0000","last_became_peered":"2024-04-12T19:51:37.316891+0000","last_unstale":"2024-04-12T19:51:38.326234+0000","last_undegraded":"2024-04-12T19:51:38.326234+0000","last_fullsized":"2024-04-12T19:51:38.326234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322072+0000","last_change":"2024-04-12T19:51:36.320077+0000","last_active":"2024-04-12T19:51:38.322072+0000","last_peered":"2024-04-12T19:51:38.322072+0000","last_clean":"2024-04-12T19:51:38.322072+0000","last_became_active":"2024-04-12T19:51:36.319763+0000","last_became_peered":"2024-04-12T19:51:36.319763+0000","last_unstale":"2024-04-12T19:51:38.322072+0000","last_undegraded":"2024-04-12T19:51:38.322072+0000","last_fullsized":"2024-04-12T19:51:38.322072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:26:59.793420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328222+0000","last_change":"2024-04-12T19:51:37.318524+0000","last_active":"2024-04-12T19:51:38.328222+0000","last_peered":"2024-04-12T19:51:38.328222+0000","last_clean":"2024-04-12T19:51:38.328222+0000","last_became_active":"2024-04-12T19:51:37.318222+0000","last_became_peered":"2024-04-12T19:51:37.318222+0000","last_unstale":"2024-04-12T19:51:38.328222+0000","last_undegraded":"2024-04-12T19:51:38.328222+0000","last_fullsized":"2024-04-12T19:51:38.328222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:10:43.194581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608026+0000","last_change":"2024-04-12T19:51:36.315549+0000","last_active":"2024-04-12T19:51:39.608026+0000","last_peered":"2024-04-12T19:51:39.608026+0000","last_clean":"2024-04-12T19:51:39.608026+0000","last_became_active":"2024-04-12T19:51:36.314815+0000","last_became_peered":"2024-04-12T19:51:36.314815+0000","last_unstale":"2024-04-12T19:51:39.608026+0000","last_undegraded":"2024-04-12T19:51:39.608026+0000","last_fullsized":"2024-04-12T19:51:39.608026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598097+0000","last_change":"2024-04-12T19:51:37.319187+0000","last_active":"2024-04-12T19:51:39.598097+0000","last_peered":"2024-04-12T19:51:39.598097+0000","last_clean":"2024-04-12T19:51:39.598097+0000","last_became_active":"2024-04-12T19:51:37.318833+0000","last_became_peered":"2024-04-12T19:51:37.318833+0000","last_unstale":"2024-04-12T19:51:39.598097+0000","last_undegraded":"2024-04-12T19:51:39.598097+0000","last_fullsized":"2024-04-12T19:51:39.598097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326973+0000","last_change":"2024-04-12T19:51:36.327171+0000","last_active":"2024-04-12T19:51:38.326973+0000","last_peered":"2024-04-12T19:51:38.326973+0000","last_clean":"2024-04-12T19:51:38.326973+0000","last_became_active":"2024-04-12T19:51:36.326818+0000","last_became_peered":"2024-04-12T19:51:36.326818+0000","last_unstale":"2024-04-12T19:51:38.326973+0000","last_undegraded":"2024-04-12T19:51:38.326973+0000","last_fullsized":"2024-04-12T19:51:38.326973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:24:03.004664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326833+0000","last_change":"2024-04-12T19:51:37.314795+0000","last_active":"2024-04-12T19:51:38.326833+0000","last_peered":"2024-04-12T19:51:38.326833+0000","last_clean":"2024-04-12T19:51:38.326833+0000","last_became_active":"2024-04-12T19:51:37.314476+0000","last_became_peered":"2024-04-12T19:51:37.314476+0000","last_unstale":"2024-04-12T19:51:38.326833+0000","last_undegraded":"2024-04-12T19:51:38.326833+0000","last_fullsized":"2024-04-12T19:51:38.326833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641698+0000","last_change":"2024-04-12T19:51:36.315615+0000","last_active":"2024-04-12T19:51:39.641698+0000","last_peered":"2024-04-12T19:51:39.641698+0000","last_clean":"2024-04-12T19:51:39.641698+0000","last_became_active":"2024-04-12T19:51:36.315325+0000","last_became_peered":"2024-04-12T19:51:36.315325+0000","last_unstale":"2024-04-12T19:51:39.641698+0000","last_undegraded":"2024-04-12T19:51:39.641698+0000","last_fullsized":"2024-04-12T19:51:39.641698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598064+0000","last_change":"2024-04-12T19:51:37.318458+0000","last_active":"2024-04-12T19:51:39.598064+0000","last_peered":"2024-04-12T19:51:39.598064+0000","last_clean":"2024-04-12T19:51:39.598064+0000","last_became_active":"2024-04-12T19:51:37.318151+0000","last_became_peered":"2024-04-12T19:51:37.318151+0000","last_unstale":"2024-04-12T19:51:39.598064+0000","last_undegraded":"2024-04-12T19:51:39.598064+0000","last_fullsized":"2024-04-12T19:51:39.598064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642177+0000","last_change":"2024-04-12T19:51:36.326548+0000","last_active":"2024-04-12T19:51:39.642177+0000","last_peered":"2024-04-12T19:51:39.642177+0000","last_clean":"2024-04-12T19:51:39.642177+0000","last_became_active":"2024-04-12T19:51:36.326046+0000","last_became_peered":"2024-04-12T19:51:36.326046+0000","last_unstale":"2024-04-12T19:51:39.642177+0000","last_undegraded":"2024-04-12T19:51:39.642177+0000","last_fullsized":"2024-04-12T19:51:39.642177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:49.798145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642346+0000","last_change":"2024-04-12T19:51:37.323108+0000","last_active":"2024-04-12T19:51:39.642346+0000","last_peered":"2024-04-12T19:51:39.642346+0000","last_clean":"2024-04-12T19:51:39.642346+0000","last_became_active":"2024-04-12T19:51:37.322777+0000","last_became_peered":"2024-04-12T19:51:37.322777+0000","last_unstale":"2024-04-12T19:51:39.642346+0000","last_undegraded":"2024-04-12T19:51:39.642346+0000","last_fullsized":"2024-04-12T19:51:39.642346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:41.414341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321166+0000","last_change":"2024-04-12T19:51:36.321134+0000","last_active":"2024-04-12T19:51:38.321166+0000","last_peered":"2024-04-12T19:51:38.321166+0000","last_clean":"2024-04-12T19:51:38.321166+0000","last_became_active":"2024-04-12T19:51:36.320811+0000","last_became_peered":"2024-04-12T19:51:36.320811+0000","last_unstale":"2024-04-12T19:51:38.321166+0000","last_undegraded":"2024-04-12T19:51:38.321166+0000","last_fullsized":"2024-04-12T19:51:38.321166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:45.437273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327966+0000","last_change":"2024-04-12T19:51:37.331351+0000","last_active":"2024-04-12T19:51:38.327966+0000","last_peered":"2024-04-12T19:51:38.327966+0000","last_clean":"2024-04-12T19:51:38.327966+0000","last_became_active":"2024-04-12T19:51:37.330895+0000","last_became_peered":"2024-04-12T19:51:37.330895+0000","last_unstale":"2024-04-12T19:51:38.327966+0000","last_undegraded":"2024-04-12T19:51:38.327966+0000","last_fullsized":"2024-04-12T19:51:38.327966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:44:51.414187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598827+0000","last_change":"2024-04-12T19:51:36.327741+0000","last_active":"2024-04-12T19:51:39.598827+0000","last_peered":"2024-04-12T19:51:39.598827+0000","last_clean":"2024-04-12T19:51:39.598827+0000","last_became_active":"2024-04-12T19:51:36.327226+0000","last_became_peered":"2024-04-12T19:51:36.327226+0000","last_unstale":"2024-04-12T19:51:39.598827+0000","last_undegraded":"2024-04-12T19:51:39.598827+0000","last_fullsized":"2024-04-12T19:51:39.598827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322019+0000","last_change":"2024-04-12T19:51:37.317114+0000","last_active":"2024-04-12T19:51:38.322019+0000","last_peered":"2024-04-12T19:51:38.322019+0000","last_clean":"2024-04-12T19:51:38.322019+0000","last_became_active":"2024-04-12T19:51:37.316821+0000","last_became_peered":"2024-04-12T19:51:37.316821+0000","last_unstale":"2024-04-12T19:51:38.322019+0000","last_undegraded":"2024-04-12T19:51:38.322019+0000","last_fullsized":"2024-04-12T19:51:38.322019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:25:58.922282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328106+0000","last_change":"2024-04-12T19:51:36.324717+0000","last_active":"2024-04-12T19:51:38.328106+0000","last_peered":"2024-04-12T19:51:38.328106+0000","last_clean":"2024-04-12T19:51:38.328106+0000","last_became_active":"2024-04-12T19:51:36.324437+0000","last_became_peered":"2024-04-12T19:51:36.324437+0000","last_unstale":"2024-04-12T19:51:38.328106+0000","last_undegraded":"2024-04-12T19:51:38.328106+0000","last_fullsized":"2024-04-12T19:51:38.328106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:14:03.050243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328579+0000","last_change":"2024-04-12T19:51:37.316433+0000","last_active":"2024-04-12T19:51:38.328579+0000","last_peered":"2024-04-12T19:51:38.328579+0000","last_clean":"2024-04-12T19:51:38.328579+0000","last_became_active":"2024-04-12T19:51:37.316089+0000","last_became_peered":"2024-04-12T19:51:37.316089+0000","last_unstale":"2024-04-12T19:51:38.328579+0000","last_undegraded":"2024-04-12T19:51:38.328579+0000","last_fullsized":"2024-04-12T19:51:38.328579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:26:38.058351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.599019+0000","last_change":"2024-04-12T19:51:36.331177+0000","last_active":"2024-04-12T19:51:39.599019+0000","last_peered":"2024-04-12T19:51:39.599019+0000","last_clean":"2024-04-12T19:51:39.599019+0000","last_became_active":"2024-04-12T19:51:36.329431+0000","last_became_peered":"2024-04-12T19:51:36.329431+0000","last_unstale":"2024-04-12T19:51:39.599019+0000","last_undegraded":"2024-04-12T19:51:39.599019+0000","last_fullsized":"2024-04-12T19:51:39.599019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607682+0000","last_change":"2024-04-12T19:51:37.319502+0000","last_active":"2024-04-12T19:51:39.607682+0000","last_peered":"2024-04-12T19:51:39.607682+0000","last_clean":"2024-04-12T19:51:39.607682+0000","last_became_active":"2024-04-12T19:51:37.318937+0000","last_became_peered":"2024-04-12T19:51:37.318937+0000","last_unstale":"2024-04-12T19:51:39.607682+0000","last_undegraded":"2024-04-12T19:51:39.607682+0000","last_fullsized":"2024-04-12T19:51:39.607682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:13.204204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598718+0000","last_change":"2024-04-12T19:51:36.321573+0000","last_active":"2024-04-12T19:51:39.598718+0000","last_peered":"2024-04-12T19:51:39.598718+0000","last_clean":"2024-04-12T19:51:39.598718+0000","last_became_active":"2024-04-12T19:51:36.321273+0000","last_became_peered":"2024-04-12T19:51:36.321273+0000","last_unstale":"2024-04-12T19:51:39.598718+0000","last_undegraded":"2024-04-12T19:51:39.598718+0000","last_fullsized":"2024-04-12T19:51:39.598718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540899+0000","last_change":"2024-04-12T19:51:37.320875+0000","last_active":"2024-04-12T19:51:38.540899+0000","last_peered":"2024-04-12T19:51:38.540899+0000","last_clean":"2024-04-12T19:51:38.540899+0000","last_became_active":"2024-04-12T19:51:37.320139+0000","last_became_peered":"2024-04-12T19:51:37.320139+0000","last_unstale":"2024-04-12T19:51:38.540899+0000","last_undegraded":"2024-04-12T19:51:38.540899+0000","last_fullsized":"2024-04-12T19:51:38.540899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:40:58.067392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598509+0000","last_change":"2024-04-12T19:51:36.323335+0000","last_active":"2024-04-12T19:51:39.598509+0000","last_peered":"2024-04-12T19:51:39.598509+0000","last_clean":"2024-04-12T19:51:39.598509+0000","last_became_active":"2024-04-12T19:51:36.323010+0000","last_became_peered":"2024-04-12T19:51:36.323010+0000","last_unstale":"2024-04-12T19:51:39.598509+0000","last_undegraded":"2024-04-12T19:51:39.598509+0000","last_fullsized":"2024-04-12T19:51:39.598509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540055+0000","last_change":"2024-04-12T19:51:37.327527+0000","last_active":"2024-04-12T19:51:38.540055+0000","last_peered":"2024-04-12T19:51:38.540055+0000","last_clean":"2024-04-12T19:51:38.540055+0000","last_became_active":"2024-04-12T19:51:37.327226+0000","last_became_peered":"2024-04-12T19:51:37.327226+0000","last_unstale":"2024-04-12T19:51:38.540055+0000","last_undegraded":"2024-04-12T19:51:38.540055+0000","last_fullsized":"2024-04-12T19:51:38.540055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:30.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641881+0000","last_change":"2024-04-12T19:51:36.315742+0000","last_active":"2024-04-12T19:51:39.641881+0000","last_peered":"2024-04-12T19:51:39.641881+0000","last_clean":"2024-04-12T19:51:39.641881+0000","last_became_active":"2024-04-12T19:51:36.315459+0000","last_became_peered":"2024-04-12T19:51:36.315459+0000","last_unstale":"2024-04-12T19:51:39.641881+0000","last_undegraded":"2024-04-12T19:51:39.641881+0000","last_fullsized":"2024-04-12T19:51:39.641881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321585+0000","last_change":"2024-04-12T19:51:37.322210+0000","last_active":"2024-04-12T19:51:38.321585+0000","last_peered":"2024-04-12T19:51:38.321585+0000","last_clean":"2024-04-12T19:51:38.321585+0000","last_became_active":"2024-04-12T19:51:37.321271+0000","last_became_peered":"2024-04-12T19:51:37.321271+0000","last_unstale":"2024-04-12T19:51:38.321585+0000","last_undegraded":"2024-04-12T19:51:38.321585+0000","last_fullsized":"2024-04-12T19:51:38.321585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:21:58.716784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540066+0000","last_change":"2024-04-12T19:51:36.324603+0000","last_active":"2024-04-12T19:51:38.540066+0000","last_peered":"2024-04-12T19:51:38.540066+0000","last_clean":"2024-04-12T19:51:38.540066+0000","last_became_active":"2024-04-12T19:51:36.322196+0000","last_became_peered":"2024-04-12T19:51:36.322196+0000","last_unstale":"2024-04-12T19:51:38.540066+0000","last_undegraded":"2024-04-12T19:51:38.540066+0000","last_fullsized":"2024-04-12T19:51:38.540066+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608151+0000","last_change":"2024-04-12T19:51:37.319380+0000","last_active":"2024-04-12T19:51:39.608151+0000","last_peered":"2024-04-12T19:51:39.608151+0000","last_clean":"2024-04-12T19:51:39.608151+0000","last_became_active":"2024-04-12T19:51:37.317927+0000","last_became_peered":"2024-04-12T19:51:37.317927+0000","last_unstale":"2024-04-12T19:51:39.608151+0000","last_undegraded":"2024-04-12T19:51:39.608151+0000","last_fullsized":"2024-04-12T19:51:39.608151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322279+0000","last_change":"2024-04-12T19:51:36.320886+0000","last_active":"2024-04-12T19:51:38.322279+0000","last_peered":"2024-04-12T19:51:38.322279+0000","last_clean":"2024-04-12T19:51:38.322279+0000","last_became_active":"2024-04-12T19:51:36.320612+0000","last_became_peered":"2024-04-12T19:51:36.320612+0000","last_unstale":"2024-04-12T19:51:38.322279+0000","last_undegraded":"2024-04-12T19:51:38.322279+0000","last_fullsized":"2024-04-12T19:51:38.322279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:23.890393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641115+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:51:39.641115+0000","last_peered":"2024-04-12T19:51:39.641115+0000","last_clean":"2024-04-12T19:51:39.641115+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:51:39.641115+0000","last_undegraded":"2024-04-12T19:51:39.641115+0000","last_fullsized":"2024-04-12T19:51:39.641115+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":742422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":153106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":152248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":741910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:51:46.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:51:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.036+0000 7f892ee00640 1 Processor -- start 2024-04-12T19:51:47.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.036+0000 7f892ee00640 1 -- start start 2024-04-12T19:51:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892ee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892809aa40 0x7f892809ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f89280d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89280b61a0 con 0x7f892815c7a0 2024-04-12T19:51:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892ee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f892809b3d0 con 0x7f892809aa40 2024-04-12T19:51:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89280b6480 con 0x7f89280cfe70 2024-04-12T19:51:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892809aa40 0x7f892809ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892809aa40 0x7f892809ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58214/0 (socket says 172.21.15.32:58214) 2024-04-12T19:51:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.038+0000 7f892d000640 1 -- 172.21.15.32:0/3385086337 learned_addr learned my addr 172.21.15.32:0/3385086337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892e400640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f89280d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892d000640 1 -- 172.21.15.32:0/3385086337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 msgr2=0x7f89280d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892d000640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f89280d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892d000640 1 -- 172.21.15.32:0/3385086337 >> v1:172.21.15.169:6789/0 conn(0x7f892815c7a0 legacy=0x7f89280b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892d000640 1 -- 172.21.15.32:0/3385086337 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89280b6760 con 0x7f892809aa40 2024-04-12T19:51:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.039+0000 7f892e400640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f89280d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.040+0000 7f892d000640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892809ae90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7585a1b05a5fcf50 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.040+0000 7f8927e00640 1 -- 172.21.15.32:0/3385086337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914035020 con 0x7f892809aa40 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.040+0000 7f8927e00640 1 -- 172.21.15.32:0/3385086337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914031640 con 0x7f892809aa40 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.040+0000 7f8927e00640 1 -- 172.21.15.32:0/3385086337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891402d040 con 0x7f892809aa40 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 -- 172.21.15.32:0/3385086337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 msgr2=0x7f892809ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892809ae90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 -- 172.21.15.32:0/3385086337 shutdown_connections 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f89280d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 --2- 172.21.15.32:0/3385086337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892809ae90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 -- 172.21.15.32:0/3385086337 >> 172.21.15.32:0/3385086337 conn(0x7f89280b40a0 msgr2=0x7f89280b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 -- 172.21.15.32:0/3385086337 shutdown_connections 2024-04-12T19:51:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.041+0000 7f892ee00640 1 -- 172.21.15.32:0/3385086337 wait complete. 2024-04-12T19:51:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.043+0000 7f892ee00640 1 Processor -- start 2024-04-12T19:51:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.043+0000 7f892ee00640 1 -- start start 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f8928154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f892815c7a0 0x7f89280d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89280b5ab0 con 0x7f892809aa40 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f89280b3520 con 0x7f892815c7a0 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f892815cb80 con 0x7f89280cfe70 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58230/0 (socket says 172.21.15.32:58230) 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.044+0000 7f892da00640 1 -- 172.21.15.32:0/2132043229 learned_addr learned my addr 172.21.15.32:0/2132043229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 -- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 msgr2=0x7f8928154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f8928154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 -- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f892815c7a0 msgr2=0x7f89280d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f892815c7a0 0x7f89280d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 -- 172.21.15.32:0/2132043229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8928150430 con 0x7f892809aa40 2024-04-12T19:51:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f892da00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b3f0 tx=0x7f891c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c015020 con 0x7f892809aa40 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c011830 con 0x7f892809aa40 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.045+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c011c20 con 0x7f892809aa40 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.046+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892815da30 con 0x7f892809aa40 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.046+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892815e290 con 0x7f892809aa40 2024-04-12T19:51:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.047+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89280755c0 con 0x7f892809aa40 2024-04-12T19:51:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.048+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f891c018020 con 0x7f892809aa40 2024-04-12T19:51:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.049+0000 7f8925600640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 0x7f890806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.049+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f891c08b770 con 0x7f892809aa40 2024-04-12T19:51:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.049+0000 7f892d000640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 0x7f890806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.050+0000 7f892d000640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 0x7f890806ee00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8914036000 tx=0x7f8914030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.053+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f891c01f030 con 0x7f892809aa40 2024-04-12T19:51:47.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.255+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f892815ddf0 con 0x7f890806c940 2024-04-12T19:51:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.265+0000 7f8925600640 1 -- 172.21.15.32:0/2132043229 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f8914031670 con 0x7f890806c940 2024-04-12T19:51:47.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:51:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 msgr2=0x7f890806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 0x7f890806ee00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8914036000 tx=0x7f8914030000 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 msgr2=0x7f892814fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b3f0 tx=0x7f891c00b8b0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 shutdown_connections 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f890806c940 0x7f890806ee00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f892815c7a0 0x7f89280d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89280cfe70 0x7f8928154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.271+0000 7f892ee00640 1 --2- 172.21.15.32:0/2132043229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892809aa40 0x7f892814fef0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.272+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 >> 172.21.15.32:0/2132043229 conn(0x7f89280b40a0 msgr2=0x7f892815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.272+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 shutdown_connections 2024-04-12T19:51:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.272+0000 7f892ee00640 1 -- 172.21.15.32:0/2132043229 wait complete. 2024-04-12T19:51:47.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-12T19:51:45.651072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237092,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737628,"statfs":{"total":773094113280,"available":772851331072,"internally_reserved":0,"allocated":4366336,"data_stored":2397332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.369651"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640727+0000","last_change":"2024-04-12T19:51:37.327897+0000","last_active":"2024-04-12T19:51:39.640727+0000","last_peered":"2024-04-12T19:51:39.640727+0000","last_clean":"2024-04-12T19:51:39.640727+0000","last_became_active":"2024-04-12T19:51:37.325584+0000","last_became_peered":"2024-04-12T19:51:37.325584+0000","last_unstale":"2024-04-12T19:51:39.640727+0000","last_undegraded":"2024-04-12T19:51:39.640727+0000","last_fullsized":"2024-04-12T19:51:39.640727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327299+0000","last_change":"2024-04-12T19:51:37.313127+0000","last_active":"2024-04-12T19:51:38.327299+0000","last_peered":"2024-04-12T19:51:38.327299+0000","last_clean":"2024-04-12T19:51:38.327299+0000","last_became_active":"2024-04-12T19:51:37.312842+0000","last_became_peered":"2024-04-12T19:51:37.312842+0000","last_unstale":"2024-04-12T19:51:38.327299+0000","last_undegraded":"2024-04-12T19:51:38.327299+0000","last_fullsized":"2024-04-12T19:51:38.327299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641376+0000","last_change":"2024-04-12T19:51:37.336313+0000","last_active":"2024-04-12T19:51:39.641376+0000","last_peered":"2024-04-12T19:51:39.641376+0000","last_clean":"2024-04-12T19:51:39.641376+0000","last_became_active":"2024-04-12T19:51:37.336041+0000","last_became_peered":"2024-04-12T19:51:37.336041+0000","last_unstale":"2024-04-12T19:51:39.641376+0000","last_undegraded":"2024-04-12T19:51:39.641376+0000","last_fullsized":"2024-04-12T19:51:39.641376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:45.554666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540672+0000","last_change":"2024-04-12T19:51:37.342853+0000","last_active":"2024-04-12T19:51:38.540672+0000","last_peered":"2024-04-12T19:51:38.540672+0000","last_clean":"2024-04-12T19:51:38.540672+0000","last_became_active":"2024-04-12T19:51:37.342521+0000","last_became_peered":"2024-04-12T19:51:37.342521+0000","last_unstale":"2024-04-12T19:51:38.540672+0000","last_undegraded":"2024-04-12T19:51:38.540672+0000","last_fullsized":"2024-04-12T19:51:38.540672+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:30:46.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326477+0000","last_change":"2024-04-12T19:51:37.324560+0000","last_active":"2024-04-12T19:51:38.326477+0000","last_peered":"2024-04-12T19:51:38.326477+0000","last_clean":"2024-04-12T19:51:38.326477+0000","last_became_active":"2024-04-12T19:51:37.324249+0000","last_became_peered":"2024-04-12T19:51:37.324249+0000","last_unstale":"2024-04-12T19:51:38.326477+0000","last_undegraded":"2024-04-12T19:51:38.326477+0000","last_fullsized":"2024-04-12T19:51:38.326477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327988+0000","last_change":"2024-04-12T19:51:37.335976+0000","last_active":"2024-04-12T19:51:38.327988+0000","last_peered":"2024-04-12T19:51:38.327988+0000","last_clean":"2024-04-12T19:51:38.327988+0000","last_became_active":"2024-04-12T19:51:37.335672+0000","last_became_peered":"2024-04-12T19:51:37.335672+0000","last_unstale":"2024-04-12T19:51:38.327988+0000","last_undegraded":"2024-04-12T19:51:38.327988+0000","last_fullsized":"2024-04-12T19:51:38.327988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:11:45.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642271+0000","last_change":"2024-04-12T19:51:37.326877+0000","last_active":"2024-04-12T19:51:39.642271+0000","last_peered":"2024-04-12T19:51:39.642271+0000","last_clean":"2024-04-12T19:51:39.642271+0000","last_became_active":"2024-04-12T19:51:37.326158+0000","last_became_peered":"2024-04-12T19:51:37.326158+0000","last_unstale":"2024-04-12T19:51:39.642271+0000","last_undegraded":"2024-04-12T19:51:39.642271+0000","last_fullsized":"2024-04-12T19:51:39.642271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:30.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642350+0000","last_change":"2024-04-12T19:51:37.335344+0000","last_active":"2024-04-12T19:51:39.642350+0000","last_peered":"2024-04-12T19:51:39.642350+0000","last_clean":"2024-04-12T19:51:39.642350+0000","last_became_active":"2024-04-12T19:51:37.328030+0000","last_became_peered":"2024-04-12T19:51:37.328030+0000","last_unstale":"2024-04-12T19:51:39.642350+0000","last_undegraded":"2024-04-12T19:51:39.642350+0000","last_fullsized":"2024-04-12T19:51:39.642350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:46.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642013+0000","last_change":"2024-04-12T19:51:37.315564+0000","last_active":"2024-04-12T19:51:39.642013+0000","last_peered":"2024-04-12T19:51:39.642013+0000","last_clean":"2024-04-12T19:51:39.642013+0000","last_became_active":"2024-04-12T19:51:37.315178+0000","last_became_peered":"2024-04-12T19:51:37.315178+0000","last_unstale":"2024-04-12T19:51:39.642013+0000","last_undegraded":"2024-04-12T19:51:39.642013+0000","last_fullsized":"2024-04-12T19:51:39.642013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:57.161489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541316+0000","last_change":"2024-04-12T19:51:37.301078+0000","last_active":"2024-04-12T19:51:38.541316+0000","last_peered":"2024-04-12T19:51:38.541316+0000","last_clean":"2024-04-12T19:51:38.541316+0000","last_became_active":"2024-04-12T19:51:37.300553+0000","last_became_peered":"2024-04-12T19:51:37.300553+0000","last_unstale":"2024-04-12T19:51:38.541316+0000","last_undegraded":"2024-04-12T19:51:38.541316+0000","last_fullsized":"2024-04-12T19:51:38.541316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:39:03.668325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322081+0000","last_change":"2024-04-12T19:51:37.313139+0000","last_active":"2024-04-12T19:51:38.322081+0000","last_peered":"2024-04-12T19:51:38.322081+0000","last_clean":"2024-04-12T19:51:38.322081+0000","last_became_active":"2024-04-12T19:51:37.312638+0000","last_became_peered":"2024-04-12T19:51:37.312638+0000","last_unstale":"2024-04-12T19:51:38.322081+0000","last_undegraded":"2024-04-12T19:51:38.322081+0000","last_fullsized":"2024-04-12T19:51:38.322081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:45.128642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640767+0000","last_change":"2024-04-12T19:51:37.320540+0000","last_active":"2024-04-12T19:51:39.640767+0000","last_peered":"2024-04-12T19:51:39.640767+0000","last_clean":"2024-04-12T19:51:39.640767+0000","last_became_active":"2024-04-12T19:51:37.320205+0000","last_became_peered":"2024-04-12T19:51:37.320205+0000","last_unstale":"2024-04-12T19:51:39.640767+0000","last_undegraded":"2024-04-12T19:51:39.640767+0000","last_fullsized":"2024-04-12T19:51:39.640767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:38.119800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598554+0000","last_change":"2024-04-12T19:51:37.322595+0000","last_active":"2024-04-12T19:51:39.598554+0000","last_peered":"2024-04-12T19:51:39.598554+0000","last_clean":"2024-04-12T19:51:39.598554+0000","last_became_active":"2024-04-12T19:51:37.321210+0000","last_became_peered":"2024-04-12T19:51:37.321210+0000","last_unstale":"2024-04-12T19:51:39.598554+0000","last_undegraded":"2024-04-12T19:51:39.598554+0000","last_fullsized":"2024-04-12T19:51:39.598554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540450+0000","last_change":"2024-04-12T19:51:37.301269+0000","last_active":"2024-04-12T19:51:38.540450+0000","last_peered":"2024-04-12T19:51:38.540450+0000","last_clean":"2024-04-12T19:51:38.540450+0000","last_became_active":"2024-04-12T19:51:37.300540+0000","last_became_peered":"2024-04-12T19:51:37.300540+0000","last_unstale":"2024-04-12T19:51:38.540450+0000","last_undegraded":"2024-04-12T19:51:38.540450+0000","last_fullsized":"2024-04-12T19:51:38.540450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:19:32.067053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607685+0000","last_change":"2024-04-12T19:51:37.329994+0000","last_active":"2024-04-12T19:51:39.607685+0000","last_peered":"2024-04-12T19:51:39.607685+0000","last_clean":"2024-04-12T19:51:39.607685+0000","last_became_active":"2024-04-12T19:51:37.329685+0000","last_became_peered":"2024-04-12T19:51:37.329685+0000","last_unstale":"2024-04-12T19:51:39.607685+0000","last_undegraded":"2024-04-12T19:51:39.607685+0000","last_fullsized":"2024-04-12T19:51:39.607685+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607696+0000","last_change":"2024-04-12T19:51:37.318607+0000","last_active":"2024-04-12T19:51:39.607696+0000","last_peered":"2024-04-12T19:51:39.607696+0000","last_clean":"2024-04-12T19:51:39.607696+0000","last_became_active":"2024-04-12T19:51:37.318119+0000","last_became_peered":"2024-04-12T19:51:37.318119+0000","last_unstale":"2024-04-12T19:51:39.607696+0000","last_undegraded":"2024-04-12T19:51:39.607696+0000","last_fullsized":"2024-04-12T19:51:39.607696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321833+0000","last_change":"2024-04-12T19:51:37.322110+0000","last_active":"2024-04-12T19:51:38.321833+0000","last_peered":"2024-04-12T19:51:38.321833+0000","last_clean":"2024-04-12T19:51:38.321833+0000","last_became_active":"2024-04-12T19:51:37.321785+0000","last_became_peered":"2024-04-12T19:51:37.321785+0000","last_unstale":"2024-04-12T19:51:38.321833+0000","last_undegraded":"2024-04-12T19:51:38.321833+0000","last_fullsized":"2024-04-12T19:51:38.321833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:53:23.809767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327881+0000","last_change":"2024-04-12T19:51:37.326355+0000","last_active":"2024-04-12T19:51:38.327881+0000","last_peered":"2024-04-12T19:51:38.327881+0000","last_clean":"2024-04-12T19:51:38.327881+0000","last_became_active":"2024-04-12T19:51:37.326026+0000","last_became_peered":"2024-04-12T19:51:37.326026+0000","last_unstale":"2024-04-12T19:51:38.327881+0000","last_undegraded":"2024-04-12T19:51:38.327881+0000","last_fullsized":"2024-04-12T19:51:38.327881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:15:16.329928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328156+0000","last_change":"2024-04-12T19:51:37.328370+0000","last_active":"2024-04-12T19:51:38.328156+0000","last_peered":"2024-04-12T19:51:38.328156+0000","last_clean":"2024-04-12T19:51:38.328156+0000","last_became_active":"2024-04-12T19:51:37.328016+0000","last_became_peered":"2024-04-12T19:51:37.328016+0000","last_unstale":"2024-04-12T19:51:38.328156+0000","last_undegraded":"2024-04-12T19:51:38.328156+0000","last_fullsized":"2024-04-12T19:51:38.328156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:59:33.416853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598843+0000","last_change":"2024-04-12T19:51:37.319605+0000","last_active":"2024-04-12T19:51:39.598843+0000","last_peered":"2024-04-12T19:51:39.598843+0000","last_clean":"2024-04-12T19:51:39.598843+0000","last_became_active":"2024-04-12T19:51:37.319245+0000","last_became_peered":"2024-04-12T19:51:37.319245+0000","last_unstale":"2024-04-12T19:51:39.598843+0000","last_undegraded":"2024-04-12T19:51:39.598843+0000","last_fullsized":"2024-04-12T19:51:39.598843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607827+0000","last_change":"2024-04-12T19:51:37.320857+0000","last_active":"2024-04-12T19:51:39.607827+0000","last_peered":"2024-04-12T19:51:39.607827+0000","last_clean":"2024-04-12T19:51:39.607827+0000","last_became_active":"2024-04-12T19:51:37.320526+0000","last_became_peered":"2024-04-12T19:51:37.320526+0000","last_unstale":"2024-04-12T19:51:39.607827+0000","last_undegraded":"2024-04-12T19:51:39.607827+0000","last_fullsized":"2024-04-12T19:51:39.607827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641608+0000","last_change":"2024-04-12T19:51:37.325464+0000","last_active":"2024-04-12T19:51:39.641608+0000","last_peered":"2024-04-12T19:51:39.641608+0000","last_clean":"2024-04-12T19:51:39.641608+0000","last_became_active":"2024-04-12T19:51:37.325178+0000","last_became_peered":"2024-04-12T19:51:37.325178+0000","last_unstale":"2024-04-12T19:51:39.641608+0000","last_undegraded":"2024-04-12T19:51:39.641608+0000","last_fullsized":"2024-04-12T19:51:39.641608+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598096+0000","last_change":"2024-04-12T19:51:37.333175+0000","last_active":"2024-04-12T19:51:39.598096+0000","last_peered":"2024-04-12T19:51:39.598096+0000","last_clean":"2024-04-12T19:51:39.598096+0000","last_became_active":"2024-04-12T19:51:37.332867+0000","last_became_peered":"2024-04-12T19:51:37.332867+0000","last_unstale":"2024-04-12T19:51:39.598096+0000","last_undegraded":"2024-04-12T19:51:39.598096+0000","last_fullsized":"2024-04-12T19:51:39.598096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:13.204204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642594+0000","last_change":"2024-04-12T19:51:37.328365+0000","last_active":"2024-04-12T19:51:39.642594+0000","last_peered":"2024-04-12T19:51:39.642594+0000","last_clean":"2024-04-12T19:51:39.642594+0000","last_became_active":"2024-04-12T19:51:37.327327+0000","last_became_peered":"2024-04-12T19:51:37.327327+0000","last_unstale":"2024-04-12T19:51:39.642594+0000","last_undegraded":"2024-04-12T19:51:39.642594+0000","last_fullsized":"2024-04-12T19:51:39.642594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:30:46.007131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541461+0000","last_change":"2024-04-12T19:51:37.312148+0000","last_active":"2024-04-12T19:51:38.541461+0000","last_peered":"2024-04-12T19:51:38.541461+0000","last_clean":"2024-04-12T19:51:38.541461+0000","last_became_active":"2024-04-12T19:51:37.311795+0000","last_became_peered":"2024-04-12T19:51:37.311795+0000","last_unstale":"2024-04-12T19:51:38.541461+0000","last_undegraded":"2024-04-12T19:51:38.541461+0000","last_fullsized":"2024-04-12T19:51:38.541461+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:34:44.292363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541572+0000","last_change":"2024-04-12T19:51:37.316522+0000","last_active":"2024-04-12T19:51:38.541572+0000","last_peered":"2024-04-12T19:51:38.541572+0000","last_clean":"2024-04-12T19:51:38.541572+0000","last_became_active":"2024-04-12T19:51:37.316208+0000","last_became_peered":"2024-04-12T19:51:37.316208+0000","last_unstale":"2024-04-12T19:51:38.541572+0000","last_undegraded":"2024-04-12T19:51:38.541572+0000","last_fullsized":"2024-04-12T19:51:38.541572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:49:38.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326806+0000","last_change":"2024-04-12T19:51:37.308586+0000","last_active":"2024-04-12T19:51:38.326806+0000","last_peered":"2024-04-12T19:51:38.326806+0000","last_clean":"2024-04-12T19:51:38.326806+0000","last_became_active":"2024-04-12T19:51:37.306339+0000","last_became_peered":"2024-04-12T19:51:37.306339+0000","last_unstale":"2024-04-12T19:51:38.326806+0000","last_undegraded":"2024-04-12T19:51:38.326806+0000","last_fullsized":"2024-04-12T19:51:38.326806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640473+0000","last_change":"2024-04-12T19:51:37.337100+0000","last_active":"2024-04-12T19:51:39.640473+0000","last_peered":"2024-04-12T19:51:39.640473+0000","last_clean":"2024-04-12T19:51:39.640473+0000","last_became_active":"2024-04-12T19:51:37.336677+0000","last_became_peered":"2024-04-12T19:51:37.336677+0000","last_unstale":"2024-04-12T19:51:39.640473+0000","last_undegraded":"2024-04-12T19:51:39.640473+0000","last_fullsized":"2024-04-12T19:51:39.640473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608098+0000","last_change":"2024-04-12T19:51:37.301241+0000","last_active":"2024-04-12T19:51:39.608098+0000","last_peered":"2024-04-12T19:51:39.608098+0000","last_clean":"2024-04-12T19:51:39.608098+0000","last_became_active":"2024-04-12T19:51:37.300515+0000","last_became_peered":"2024-04-12T19:51:37.300515+0000","last_unstale":"2024-04-12T19:51:39.608098+0000","last_undegraded":"2024-04-12T19:51:39.608098+0000","last_fullsized":"2024-04-12T19:51:39.608098+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607767+0000","last_change":"2024-04-12T19:51:37.319225+0000","last_active":"2024-04-12T19:51:39.607767+0000","last_peered":"2024-04-12T19:51:39.607767+0000","last_clean":"2024-04-12T19:51:39.607767+0000","last_became_active":"2024-04-12T19:51:37.317746+0000","last_became_peered":"2024-04-12T19:51:37.317746+0000","last_unstale":"2024-04-12T19:51:39.607767+0000","last_undegraded":"2024-04-12T19:51:39.607767+0000","last_fullsized":"2024-04-12T19:51:39.607767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641623+0000","last_change":"2024-04-12T19:51:37.327074+0000","last_active":"2024-04-12T19:51:39.641623+0000","last_peered":"2024-04-12T19:51:39.641623+0000","last_clean":"2024-04-12T19:51:39.641623+0000","last_became_active":"2024-04-12T19:51:37.326493+0000","last_became_peered":"2024-04-12T19:51:37.326493+0000","last_unstale":"2024-04-12T19:51:39.641623+0000","last_undegraded":"2024-04-12T19:51:39.641623+0000","last_fullsized":"2024-04-12T19:51:39.641623+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321339+0000","last_change":"2024-04-12T19:51:37.332286+0000","last_active":"2024-04-12T19:51:38.321339+0000","last_peered":"2024-04-12T19:51:38.321339+0000","last_clean":"2024-04-12T19:51:38.321339+0000","last_became_active":"2024-04-12T19:51:37.330122+0000","last_became_peered":"2024-04-12T19:51:37.330122+0000","last_unstale":"2024-04-12T19:51:38.321339+0000","last_undegraded":"2024-04-12T19:51:38.321339+0000","last_fullsized":"2024-04-12T19:51:38.321339+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:20:56.060888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.599010+0000","last_change":"2024-04-12T19:51:37.322199+0000","last_active":"2024-04-12T19:51:39.599010+0000","last_peered":"2024-04-12T19:51:39.599010+0000","last_clean":"2024-04-12T19:51:39.599010+0000","last_became_active":"2024-04-12T19:51:37.321849+0000","last_became_peered":"2024-04-12T19:51:37.321849+0000","last_unstale":"2024-04-12T19:51:39.599010+0000","last_undegraded":"2024-04-12T19:51:39.599010+0000","last_fullsized":"2024-04-12T19:51:39.599010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322105+0000","last_change":"2024-04-12T19:51:36.323028+0000","last_active":"2024-04-12T19:51:38.322105+0000","last_peered":"2024-04-12T19:51:38.322105+0000","last_clean":"2024-04-12T19:51:38.322105+0000","last_became_active":"2024-04-12T19:51:36.322694+0000","last_became_peered":"2024-04-12T19:51:36.322694+0000","last_unstale":"2024-04-12T19:51:38.322105+0000","last_undegraded":"2024-04-12T19:51:38.322105+0000","last_fullsized":"2024-04-12T19:51:38.322105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:29.889791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641057+0000","last_change":"2024-04-12T19:51:37.334931+0000","last_active":"2024-04-12T19:51:39.641057+0000","last_peered":"2024-04-12T19:51:39.641057+0000","last_clean":"2024-04-12T19:51:39.641057+0000","last_became_active":"2024-04-12T19:51:37.331107+0000","last_became_peered":"2024-04-12T19:51:37.331107+0000","last_unstale":"2024-04-12T19:51:39.641057+0000","last_undegraded":"2024-04-12T19:51:39.641057+0000","last_fullsized":"2024-04-12T19:51:39.641057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541600+0000","last_change":"2024-04-12T19:51:36.325298+0000","last_active":"2024-04-12T19:51:38.541600+0000","last_peered":"2024-04-12T19:51:38.541600+0000","last_clean":"2024-04-12T19:51:38.541600+0000","last_became_active":"2024-04-12T19:51:36.324222+0000","last_became_peered":"2024-04-12T19:51:36.324222+0000","last_unstale":"2024-04-12T19:51:38.541600+0000","last_undegraded":"2024-04-12T19:51:38.541600+0000","last_fullsized":"2024-04-12T19:51:38.541600+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:44:36.049413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640563+0000","last_change":"2024-04-12T19:51:37.322294+0000","last_active":"2024-04-12T19:51:39.640563+0000","last_peered":"2024-04-12T19:51:39.640563+0000","last_clean":"2024-04-12T19:51:39.640563+0000","last_became_active":"2024-04-12T19:51:37.321974+0000","last_became_peered":"2024-04-12T19:51:37.321974+0000","last_unstale":"2024-04-12T19:51:39.640563+0000","last_undegraded":"2024-04-12T19:51:39.640563+0000","last_fullsized":"2024-04-12T19:51:39.640563+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328355+0000","last_change":"2024-04-12T19:51:36.331707+0000","last_active":"2024-04-12T19:51:38.328355+0000","last_peered":"2024-04-12T19:51:38.328355+0000","last_clean":"2024-04-12T19:51:38.328355+0000","last_became_active":"2024-04-12T19:51:36.331198+0000","last_became_peered":"2024-04-12T19:51:36.331198+0000","last_unstale":"2024-04-12T19:51:38.328355+0000","last_undegraded":"2024-04-12T19:51:38.328355+0000","last_fullsized":"2024-04-12T19:51:38.328355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:59:18.552643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540967+0000","last_change":"2024-04-12T19:51:37.325365+0000","last_active":"2024-04-12T19:51:38.540967+0000","last_peered":"2024-04-12T19:51:38.540967+0000","last_clean":"2024-04-12T19:51:38.540967+0000","last_became_active":"2024-04-12T19:51:37.323723+0000","last_became_peered":"2024-04-12T19:51:37.323723+0000","last_unstale":"2024-04-12T19:51:38.540967+0000","last_undegraded":"2024-04-12T19:51:38.540967+0000","last_fullsized":"2024-04-12T19:51:38.540967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:31:49.343920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541024+0000","last_change":"2024-04-12T19:51:36.330056+0000","last_active":"2024-04-12T19:51:38.541024+0000","last_peered":"2024-04-12T19:51:38.541024+0000","last_clean":"2024-04-12T19:51:38.541024+0000","last_became_active":"2024-04-12T19:51:36.329657+0000","last_became_peered":"2024-04-12T19:51:36.329657+0000","last_unstale":"2024-04-12T19:51:38.541024+0000","last_undegraded":"2024-04-12T19:51:38.541024+0000","last_fullsized":"2024-04-12T19:51:38.541024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640592+0000","last_change":"2024-04-12T19:51:37.317712+0000","last_active":"2024-04-12T19:51:39.640592+0000","last_peered":"2024-04-12T19:51:39.640592+0000","last_clean":"2024-04-12T19:51:39.640592+0000","last_became_active":"2024-04-12T19:51:37.317351+0000","last_became_peered":"2024-04-12T19:51:37.317351+0000","last_unstale":"2024-04-12T19:51:39.640592+0000","last_undegraded":"2024-04-12T19:51:39.640592+0000","last_fullsized":"2024-04-12T19:51:39.640592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:42.048987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640936+0000","last_change":"2024-04-12T19:51:36.327409+0000","last_active":"2024-04-12T19:51:39.640936+0000","last_peered":"2024-04-12T19:51:39.640936+0000","last_clean":"2024-04-12T19:51:39.640936+0000","last_became_active":"2024-04-12T19:51:36.327107+0000","last_became_peered":"2024-04-12T19:51:36.327107+0000","last_unstale":"2024-04-12T19:51:39.640936+0000","last_undegraded":"2024-04-12T19:51:39.640936+0000","last_fullsized":"2024-04-12T19:51:39.640936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:40:21.315726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640739+0000","last_change":"2024-04-12T19:51:37.327878+0000","last_active":"2024-04-12T19:51:39.640739+0000","last_peered":"2024-04-12T19:51:39.640739+0000","last_clean":"2024-04-12T19:51:39.640739+0000","last_became_active":"2024-04-12T19:51:37.327592+0000","last_became_peered":"2024-04-12T19:51:37.327592+0000","last_unstale":"2024-04-12T19:51:39.640739+0000","last_undegraded":"2024-04-12T19:51:39.640739+0000","last_fullsized":"2024-04-12T19:51:39.640739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:09:10.364511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641467+0000","last_change":"2024-04-12T19:51:36.324064+0000","last_active":"2024-04-12T19:51:39.641467+0000","last_peered":"2024-04-12T19:51:39.641467+0000","last_clean":"2024-04-12T19:51:39.641467+0000","last_became_active":"2024-04-12T19:51:36.323334+0000","last_became_peered":"2024-04-12T19:51:36.323334+0000","last_unstale":"2024-04-12T19:51:39.641467+0000","last_undegraded":"2024-04-12T19:51:39.641467+0000","last_fullsized":"2024-04-12T19:51:39.641467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:18:22.074239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640556+0000","last_change":"2024-04-12T19:51:37.322281+0000","last_active":"2024-04-12T19:51:39.640556+0000","last_peered":"2024-04-12T19:51:39.640556+0000","last_clean":"2024-04-12T19:51:39.640556+0000","last_became_active":"2024-04-12T19:51:37.318930+0000","last_became_peered":"2024-04-12T19:51:37.318930+0000","last_unstale":"2024-04-12T19:51:39.640556+0000","last_undegraded":"2024-04-12T19:51:39.640556+0000","last_fullsized":"2024-04-12T19:51:39.640556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:48.497592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326879+0000","last_change":"2024-04-12T19:51:36.326080+0000","last_active":"2024-04-12T19:51:38.326879+0000","last_peered":"2024-04-12T19:51:38.326879+0000","last_clean":"2024-04-12T19:51:38.326879+0000","last_became_active":"2024-04-12T19:51:36.325817+0000","last_became_peered":"2024-04-12T19:51:36.325817+0000","last_unstale":"2024-04-12T19:51:38.326879+0000","last_undegraded":"2024-04-12T19:51:38.326879+0000","last_fullsized":"2024-04-12T19:51:38.326879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:56:03.515939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.539962+0000","last_change":"2024-04-12T19:51:37.312621+0000","last_active":"2024-04-12T19:51:38.539962+0000","last_peered":"2024-04-12T19:51:38.539962+0000","last_clean":"2024-04-12T19:51:38.539962+0000","last_became_active":"2024-04-12T19:51:37.312092+0000","last_became_peered":"2024-04-12T19:51:37.312092+0000","last_unstale":"2024-04-12T19:51:38.539962+0000","last_undegraded":"2024-04-12T19:51:38.539962+0000","last_fullsized":"2024-04-12T19:51:38.539962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:24.906891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642833+0000","last_change":"2024-04-12T19:51:36.320855+0000","last_active":"2024-04-12T19:51:39.642833+0000","last_peered":"2024-04-12T19:51:39.642833+0000","last_clean":"2024-04-12T19:51:39.642833+0000","last_became_active":"2024-04-12T19:51:36.320566+0000","last_became_peered":"2024-04-12T19:51:36.320566+0000","last_unstale":"2024-04-12T19:51:39.642833+0000","last_undegraded":"2024-04-12T19:51:39.642833+0000","last_fullsized":"2024-04-12T19:51:39.642833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321976+0000","last_change":"2024-04-12T19:51:37.328807+0000","last_active":"2024-04-12T19:51:38.321976+0000","last_peered":"2024-04-12T19:51:38.321976+0000","last_clean":"2024-04-12T19:51:38.321976+0000","last_became_active":"2024-04-12T19:51:37.323818+0000","last_became_peered":"2024-04-12T19:51:37.323818+0000","last_unstale":"2024-04-12T19:51:38.321976+0000","last_undegraded":"2024-04-12T19:51:38.321976+0000","last_fullsized":"2024-04-12T19:51:38.321976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:48:11.338822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327684+0000","last_change":"2024-04-12T19:51:36.323202+0000","last_active":"2024-04-12T19:51:38.327684+0000","last_peered":"2024-04-12T19:51:38.327684+0000","last_clean":"2024-04-12T19:51:38.327684+0000","last_became_active":"2024-04-12T19:51:36.322883+0000","last_became_peered":"2024-04-12T19:51:36.322883+0000","last_unstale":"2024-04-12T19:51:38.327684+0000","last_undegraded":"2024-04-12T19:51:38.327684+0000","last_fullsized":"2024-04-12T19:51:38.327684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T20:42:56.574639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541543+0000","last_change":"2024-04-12T19:51:37.309643+0000","last_active":"2024-04-12T19:51:38.541543+0000","last_peered":"2024-04-12T19:51:38.541543+0000","last_clean":"2024-04-12T19:51:38.541543+0000","last_became_active":"2024-04-12T19:51:37.303858+0000","last_became_peered":"2024-04-12T19:51:37.303858+0000","last_unstale":"2024-04-12T19:51:38.541543+0000","last_undegraded":"2024-04-12T19:51:38.541543+0000","last_fullsized":"2024-04-12T19:51:38.541543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:03:09.445009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327103+0000","last_change":"2024-04-12T19:51:36.327948+0000","last_active":"2024-04-12T19:51:38.327103+0000","last_peered":"2024-04-12T19:51:38.327103+0000","last_clean":"2024-04-12T19:51:38.327103+0000","last_became_active":"2024-04-12T19:51:36.327654+0000","last_became_peered":"2024-04-12T19:51:36.327654+0000","last_unstale":"2024-04-12T19:51:38.327103+0000","last_undegraded":"2024-04-12T19:51:38.327103+0000","last_fullsized":"2024-04-12T19:51:38.327103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:17:47.877823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540673+0000","last_change":"2024-04-12T19:51:37.315215+0000","last_active":"2024-04-12T19:51:38.540673+0000","last_peered":"2024-04-12T19:51:38.540673+0000","last_clean":"2024-04-12T19:51:38.540673+0000","last_became_active":"2024-04-12T19:51:37.314923+0000","last_became_peered":"2024-04-12T19:51:37.314923+0000","last_unstale":"2024-04-12T19:51:38.540673+0000","last_undegraded":"2024-04-12T19:51:38.540673+0000","last_fullsized":"2024-04-12T19:51:38.540673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:46.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321773+0000","last_change":"2024-04-12T19:51:36.304808+0000","last_active":"2024-04-12T19:51:38.321773+0000","last_peered":"2024-04-12T19:51:38.321773+0000","last_clean":"2024-04-12T19:51:38.321773+0000","last_became_active":"2024-04-12T19:51:36.304470+0000","last_became_peered":"2024-04-12T19:51:36.304470+0000","last_unstale":"2024-04-12T19:51:38.321773+0000","last_undegraded":"2024-04-12T19:51:38.321773+0000","last_fullsized":"2024-04-12T19:51:38.321773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:24.374284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321768+0000","last_change":"2024-04-12T19:51:37.312919+0000","last_active":"2024-04-12T19:51:38.321768+0000","last_peered":"2024-04-12T19:51:38.321768+0000","last_clean":"2024-04-12T19:51:38.321768+0000","last_became_active":"2024-04-12T19:51:37.312195+0000","last_became_peered":"2024-04-12T19:51:37.312195+0000","last_unstale":"2024-04-12T19:51:38.321768+0000","last_undegraded":"2024-04-12T19:51:38.321768+0000","last_fullsized":"2024-04-12T19:51:38.321768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:03:31.127824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598741+0000","last_change":"2024-04-12T19:51:36.331502+0000","last_active":"2024-04-12T19:51:39.598741+0000","last_peered":"2024-04-12T19:51:39.598741+0000","last_clean":"2024-04-12T19:51:39.598741+0000","last_became_active":"2024-04-12T19:51:36.329297+0000","last_became_peered":"2024-04-12T19:51:36.329297+0000","last_unstale":"2024-04-12T19:51:39.598741+0000","last_undegraded":"2024-04-12T19:51:39.598741+0000","last_fullsized":"2024-04-12T19:51:39.598741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598033+0000","last_change":"2024-04-12T19:51:37.320432+0000","last_active":"2024-04-12T19:51:39.598033+0000","last_peered":"2024-04-12T19:51:39.598033+0000","last_clean":"2024-04-12T19:51:39.598033+0000","last_became_active":"2024-04-12T19:51:37.319862+0000","last_became_peered":"2024-04-12T19:51:37.319862+0000","last_unstale":"2024-04-12T19:51:39.598033+0000","last_undegraded":"2024-04-12T19:51:39.598033+0000","last_fullsized":"2024-04-12T19:51:39.598033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:22:08.107072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641138+0000","last_change":"2024-04-12T19:51:36.316118+0000","last_active":"2024-04-12T19:51:39.641138+0000","last_peered":"2024-04-12T19:51:39.641138+0000","last_clean":"2024-04-12T19:51:39.641138+0000","last_became_active":"2024-04-12T19:51:36.315852+0000","last_became_peered":"2024-04-12T19:51:36.315852+0000","last_unstale":"2024-04-12T19:51:39.641138+0000","last_undegraded":"2024-04-12T19:51:39.641138+0000","last_fullsized":"2024-04-12T19:51:39.641138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:49:40.856697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.541341+0000","last_change":"2024-04-12T19:51:37.325180+0000","last_active":"2024-04-12T19:51:38.541341+0000","last_peered":"2024-04-12T19:51:38.541341+0000","last_clean":"2024-04-12T19:51:38.541341+0000","last_became_active":"2024-04-12T19:51:37.323566+0000","last_became_peered":"2024-04-12T19:51:37.323566+0000","last_unstale":"2024-04-12T19:51:38.541341+0000","last_undegraded":"2024-04-12T19:51:38.541341+0000","last_fullsized":"2024-04-12T19:51:38.541341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:58:36.162354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540061+0000","last_change":"2024-04-12T19:51:36.320973+0000","last_active":"2024-04-12T19:51:38.540061+0000","last_peered":"2024-04-12T19:51:38.540061+0000","last_clean":"2024-04-12T19:51:38.540061+0000","last_became_active":"2024-04-12T19:51:36.320688+0000","last_became_peered":"2024-04-12T19:51:36.320688+0000","last_unstale":"2024-04-12T19:51:38.540061+0000","last_undegraded":"2024-04-12T19:51:38.540061+0000","last_fullsized":"2024-04-12T19:51:38.540061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642731+0000","last_change":"2024-04-12T19:51:37.327683+0000","last_active":"2024-04-12T19:51:39.642731+0000","last_peered":"2024-04-12T19:51:39.642731+0000","last_clean":"2024-04-12T19:51:39.642731+0000","last_became_active":"2024-04-12T19:51:37.326936+0000","last_became_peered":"2024-04-12T19:51:37.326936+0000","last_unstale":"2024-04-12T19:51:39.642731+0000","last_undegraded":"2024-04-12T19:51:39.642731+0000","last_fullsized":"2024-04-12T19:51:39.642731+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:40:58.067392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.539745+0000","last_change":"2024-04-12T19:51:36.313661+0000","last_active":"2024-04-12T19:51:38.539745+0000","last_peered":"2024-04-12T19:51:38.539745+0000","last_clean":"2024-04-12T19:51:38.539745+0000","last_became_active":"2024-04-12T19:51:36.313249+0000","last_became_peered":"2024-04-12T19:51:36.313249+0000","last_unstale":"2024-04-12T19:51:38.539745+0000","last_undegraded":"2024-04-12T19:51:38.539745+0000","last_fullsized":"2024-04-12T19:51:38.539745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321330+0000","last_change":"2024-04-12T19:51:37.330768+0000","last_active":"2024-04-12T19:51:38.321330+0000","last_peered":"2024-04-12T19:51:38.321330+0000","last_clean":"2024-04-12T19:51:38.321330+0000","last_became_active":"2024-04-12T19:51:37.329678+0000","last_became_peered":"2024-04-12T19:51:37.329678+0000","last_unstale":"2024-04-12T19:51:38.321330+0000","last_undegraded":"2024-04-12T19:51:38.321330+0000","last_fullsized":"2024-04-12T19:51:38.321330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:10:30.334195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326281+0000","last_change":"2024-04-12T19:51:36.324874+0000","last_active":"2024-04-12T19:51:38.326281+0000","last_peered":"2024-04-12T19:51:38.326281+0000","last_clean":"2024-04-12T19:51:38.326281+0000","last_became_active":"2024-04-12T19:51:36.324013+0000","last_became_peered":"2024-04-12T19:51:36.324013+0000","last_unstale":"2024-04-12T19:51:38.326281+0000","last_undegraded":"2024-04-12T19:51:38.326281+0000","last_fullsized":"2024-04-12T19:51:38.326281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T23:10:23.725348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322274+0000","last_change":"2024-04-12T19:51:37.319644+0000","last_active":"2024-04-12T19:51:38.322274+0000","last_peered":"2024-04-12T19:51:38.322274+0000","last_clean":"2024-04-12T19:51:38.322274+0000","last_became_active":"2024-04-12T19:51:37.319356+0000","last_became_peered":"2024-04-12T19:51:37.319356+0000","last_unstale":"2024-04-12T19:51:38.322274+0000","last_undegraded":"2024-04-12T19:51:38.322274+0000","last_fullsized":"2024-04-12T19:51:38.322274+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:14:26.033574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328103+0000","last_change":"2024-04-12T19:51:36.332183+0000","last_active":"2024-04-12T19:51:38.328103+0000","last_peered":"2024-04-12T19:51:38.328103+0000","last_clean":"2024-04-12T19:51:38.328103+0000","last_became_active":"2024-04-12T19:51:36.331021+0000","last_became_peered":"2024-04-12T19:51:36.331021+0000","last_unstale":"2024-04-12T19:51:38.328103+0000","last_undegraded":"2024-04-12T19:51:38.328103+0000","last_fullsized":"2024-04-12T19:51:38.328103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:29:38.531795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642261+0000","last_change":"2024-04-12T19:51:37.314878+0000","last_active":"2024-04-12T19:51:39.642261+0000","last_peered":"2024-04-12T19:51:39.642261+0000","last_clean":"2024-04-12T19:51:39.642261+0000","last_became_active":"2024-04-12T19:51:37.314541+0000","last_became_peered":"2024-04-12T19:51:37.314541+0000","last_unstale":"2024-04-12T19:51:39.642261+0000","last_undegraded":"2024-04-12T19:51:39.642261+0000","last_fullsized":"2024-04-12T19:51:39.642261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:54.323339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.640369+0000","last_change":"2024-04-12T19:51:36.311352+0000","last_active":"2024-04-12T19:51:39.640369+0000","last_peered":"2024-04-12T19:51:39.640369+0000","last_clean":"2024-04-12T19:51:39.640369+0000","last_became_active":"2024-04-12T19:51:36.310541+0000","last_became_peered":"2024-04-12T19:51:36.310541+0000","last_unstale":"2024-04-12T19:51:39.640369+0000","last_undegraded":"2024-04-12T19:51:39.640369+0000","last_fullsized":"2024-04-12T19:51:39.640369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326234+0000","last_change":"2024-04-12T19:51:37.317144+0000","last_active":"2024-04-12T19:51:38.326234+0000","last_peered":"2024-04-12T19:51:38.326234+0000","last_clean":"2024-04-12T19:51:38.326234+0000","last_became_active":"2024-04-12T19:51:37.316891+0000","last_became_peered":"2024-04-12T19:51:37.316891+0000","last_unstale":"2024-04-12T19:51:38.326234+0000","last_undegraded":"2024-04-12T19:51:38.326234+0000","last_fullsized":"2024-04-12T19:51:38.326234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:00:17.133717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322072+0000","last_change":"2024-04-12T19:51:36.320077+0000","last_active":"2024-04-12T19:51:38.322072+0000","last_peered":"2024-04-12T19:51:38.322072+0000","last_clean":"2024-04-12T19:51:38.322072+0000","last_became_active":"2024-04-12T19:51:36.319763+0000","last_became_peered":"2024-04-12T19:51:36.319763+0000","last_unstale":"2024-04-12T19:51:38.322072+0000","last_undegraded":"2024-04-12T19:51:38.322072+0000","last_fullsized":"2024-04-12T19:51:38.322072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:26:59.793420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328222+0000","last_change":"2024-04-12T19:51:37.318524+0000","last_active":"2024-04-12T19:51:38.328222+0000","last_peered":"2024-04-12T19:51:38.328222+0000","last_clean":"2024-04-12T19:51:38.328222+0000","last_became_active":"2024-04-12T19:51:37.318222+0000","last_became_peered":"2024-04-12T19:51:37.318222+0000","last_unstale":"2024-04-12T19:51:38.328222+0000","last_undegraded":"2024-04-12T19:51:38.328222+0000","last_fullsized":"2024-04-12T19:51:38.328222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:10:43.194581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608026+0000","last_change":"2024-04-12T19:51:36.315549+0000","last_active":"2024-04-12T19:51:39.608026+0000","last_peered":"2024-04-12T19:51:39.608026+0000","last_clean":"2024-04-12T19:51:39.608026+0000","last_became_active":"2024-04-12T19:51:36.314815+0000","last_became_peered":"2024-04-12T19:51:36.314815+0000","last_unstale":"2024-04-12T19:51:39.608026+0000","last_undegraded":"2024-04-12T19:51:39.608026+0000","last_fullsized":"2024-04-12T19:51:39.608026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:40.397843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598097+0000","last_change":"2024-04-12T19:51:37.319187+0000","last_active":"2024-04-12T19:51:39.598097+0000","last_peered":"2024-04-12T19:51:39.598097+0000","last_clean":"2024-04-12T19:51:39.598097+0000","last_became_active":"2024-04-12T19:51:37.318833+0000","last_became_peered":"2024-04-12T19:51:37.318833+0000","last_unstale":"2024-04-12T19:51:39.598097+0000","last_undegraded":"2024-04-12T19:51:39.598097+0000","last_fullsized":"2024-04-12T19:51:39.598097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:52.583837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326973+0000","last_change":"2024-04-12T19:51:36.327171+0000","last_active":"2024-04-12T19:51:38.326973+0000","last_peered":"2024-04-12T19:51:38.326973+0000","last_clean":"2024-04-12T19:51:38.326973+0000","last_became_active":"2024-04-12T19:51:36.326818+0000","last_became_peered":"2024-04-12T19:51:36.326818+0000","last_unstale":"2024-04-12T19:51:38.326973+0000","last_undegraded":"2024-04-12T19:51:38.326973+0000","last_fullsized":"2024-04-12T19:51:38.326973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:24:03.004664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.326833+0000","last_change":"2024-04-12T19:51:37.314795+0000","last_active":"2024-04-12T19:51:38.326833+0000","last_peered":"2024-04-12T19:51:38.326833+0000","last_clean":"2024-04-12T19:51:38.326833+0000","last_became_active":"2024-04-12T19:51:37.314476+0000","last_became_peered":"2024-04-12T19:51:37.314476+0000","last_unstale":"2024-04-12T19:51:38.326833+0000","last_undegraded":"2024-04-12T19:51:38.326833+0000","last_fullsized":"2024-04-12T19:51:38.326833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:27:00.809919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641698+0000","last_change":"2024-04-12T19:51:36.315615+0000","last_active":"2024-04-12T19:51:39.641698+0000","last_peered":"2024-04-12T19:51:39.641698+0000","last_clean":"2024-04-12T19:51:39.641698+0000","last_became_active":"2024-04-12T19:51:36.315325+0000","last_became_peered":"2024-04-12T19:51:36.315325+0000","last_unstale":"2024-04-12T19:51:39.641698+0000","last_undegraded":"2024-04-12T19:51:39.641698+0000","last_fullsized":"2024-04-12T19:51:39.641698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:06:47.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598064+0000","last_change":"2024-04-12T19:51:37.318458+0000","last_active":"2024-04-12T19:51:39.598064+0000","last_peered":"2024-04-12T19:51:39.598064+0000","last_clean":"2024-04-12T19:51:39.598064+0000","last_became_active":"2024-04-12T19:51:37.318151+0000","last_became_peered":"2024-04-12T19:51:37.318151+0000","last_unstale":"2024-04-12T19:51:39.598064+0000","last_undegraded":"2024-04-12T19:51:39.598064+0000","last_fullsized":"2024-04-12T19:51:39.598064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T22:32:39.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642177+0000","last_change":"2024-04-12T19:51:36.326548+0000","last_active":"2024-04-12T19:51:39.642177+0000","last_peered":"2024-04-12T19:51:39.642177+0000","last_clean":"2024-04-12T19:51:39.642177+0000","last_became_active":"2024-04-12T19:51:36.326046+0000","last_became_peered":"2024-04-12T19:51:36.326046+0000","last_unstale":"2024-04-12T19:51:39.642177+0000","last_undegraded":"2024-04-12T19:51:39.642177+0000","last_fullsized":"2024-04-12T19:51:39.642177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:20:49.798145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.642346+0000","last_change":"2024-04-12T19:51:37.323108+0000","last_active":"2024-04-12T19:51:39.642346+0000","last_peered":"2024-04-12T19:51:39.642346+0000","last_clean":"2024-04-12T19:51:39.642346+0000","last_became_active":"2024-04-12T19:51:37.322777+0000","last_became_peered":"2024-04-12T19:51:37.322777+0000","last_unstale":"2024-04-12T19:51:39.642346+0000","last_undegraded":"2024-04-12T19:51:39.642346+0000","last_fullsized":"2024-04-12T19:51:39.642346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T06:31:41.414341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321166+0000","last_change":"2024-04-12T19:51:36.321134+0000","last_active":"2024-04-12T19:51:38.321166+0000","last_peered":"2024-04-12T19:51:38.321166+0000","last_clean":"2024-04-12T19:51:38.321166+0000","last_became_active":"2024-04-12T19:51:36.320811+0000","last_became_peered":"2024-04-12T19:51:36.320811+0000","last_unstale":"2024-04-12T19:51:38.321166+0000","last_undegraded":"2024-04-12T19:51:38.321166+0000","last_fullsized":"2024-04-12T19:51:38.321166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:32:45.437273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.327966+0000","last_change":"2024-04-12T19:51:37.331351+0000","last_active":"2024-04-12T19:51:38.327966+0000","last_peered":"2024-04-12T19:51:38.327966+0000","last_clean":"2024-04-12T19:51:38.327966+0000","last_became_active":"2024-04-12T19:51:37.330895+0000","last_became_peered":"2024-04-12T19:51:37.330895+0000","last_unstale":"2024-04-12T19:51:38.327966+0000","last_undegraded":"2024-04-12T19:51:38.327966+0000","last_fullsized":"2024-04-12T19:51:38.327966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:44:51.414187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598827+0000","last_change":"2024-04-12T19:51:36.327741+0000","last_active":"2024-04-12T19:51:39.598827+0000","last_peered":"2024-04-12T19:51:39.598827+0000","last_clean":"2024-04-12T19:51:39.598827+0000","last_became_active":"2024-04-12T19:51:36.327226+0000","last_became_peered":"2024-04-12T19:51:36.327226+0000","last_unstale":"2024-04-12T19:51:39.598827+0000","last_undegraded":"2024-04-12T19:51:39.598827+0000","last_fullsized":"2024-04-12T19:51:39.598827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322019+0000","last_change":"2024-04-12T19:51:37.317114+0000","last_active":"2024-04-12T19:51:38.322019+0000","last_peered":"2024-04-12T19:51:38.322019+0000","last_clean":"2024-04-12T19:51:38.322019+0000","last_became_active":"2024-04-12T19:51:37.316821+0000","last_became_peered":"2024-04-12T19:51:37.316821+0000","last_unstale":"2024-04-12T19:51:38.322019+0000","last_undegraded":"2024-04-12T19:51:38.322019+0000","last_fullsized":"2024-04-12T19:51:38.322019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:25:58.922282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328106+0000","last_change":"2024-04-12T19:51:36.324717+0000","last_active":"2024-04-12T19:51:38.328106+0000","last_peered":"2024-04-12T19:51:38.328106+0000","last_clean":"2024-04-12T19:51:38.328106+0000","last_became_active":"2024-04-12T19:51:36.324437+0000","last_became_peered":"2024-04-12T19:51:36.324437+0000","last_unstale":"2024-04-12T19:51:38.328106+0000","last_undegraded":"2024-04-12T19:51:38.328106+0000","last_fullsized":"2024-04-12T19:51:38.328106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:14:03.050243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.328579+0000","last_change":"2024-04-12T19:51:37.316433+0000","last_active":"2024-04-12T19:51:38.328579+0000","last_peered":"2024-04-12T19:51:38.328579+0000","last_clean":"2024-04-12T19:51:38.328579+0000","last_became_active":"2024-04-12T19:51:37.316089+0000","last_became_peered":"2024-04-12T19:51:37.316089+0000","last_unstale":"2024-04-12T19:51:38.328579+0000","last_undegraded":"2024-04-12T19:51:38.328579+0000","last_fullsized":"2024-04-12T19:51:38.328579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T01:26:38.058351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.599019+0000","last_change":"2024-04-12T19:51:36.331177+0000","last_active":"2024-04-12T19:51:39.599019+0000","last_peered":"2024-04-12T19:51:39.599019+0000","last_clean":"2024-04-12T19:51:39.599019+0000","last_became_active":"2024-04-12T19:51:36.329431+0000","last_became_peered":"2024-04-12T19:51:36.329431+0000","last_unstale":"2024-04-12T19:51:39.599019+0000","last_undegraded":"2024-04-12T19:51:39.599019+0000","last_fullsized":"2024-04-12T19:51:39.599019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:22:53.306841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.607682+0000","last_change":"2024-04-12T19:51:37.319502+0000","last_active":"2024-04-12T19:51:39.607682+0000","last_peered":"2024-04-12T19:51:39.607682+0000","last_clean":"2024-04-12T19:51:39.607682+0000","last_became_active":"2024-04-12T19:51:37.318937+0000","last_became_peered":"2024-04-12T19:51:37.318937+0000","last_unstale":"2024-04-12T19:51:39.607682+0000","last_undegraded":"2024-04-12T19:51:39.607682+0000","last_fullsized":"2024-04-12T19:51:39.607682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T00:14:13.204204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598718+0000","last_change":"2024-04-12T19:51:36.321573+0000","last_active":"2024-04-12T19:51:39.598718+0000","last_peered":"2024-04-12T19:51:39.598718+0000","last_clean":"2024-04-12T19:51:39.598718+0000","last_became_active":"2024-04-12T19:51:36.321273+0000","last_became_peered":"2024-04-12T19:51:36.321273+0000","last_unstale":"2024-04-12T19:51:39.598718+0000","last_undegraded":"2024-04-12T19:51:39.598718+0000","last_fullsized":"2024-04-12T19:51:39.598718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:42:37.103302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540899+0000","last_change":"2024-04-12T19:51:37.320875+0000","last_active":"2024-04-12T19:51:38.540899+0000","last_peered":"2024-04-12T19:51:38.540899+0000","last_clean":"2024-04-12T19:51:38.540899+0000","last_became_active":"2024-04-12T19:51:37.320139+0000","last_became_peered":"2024-04-12T19:51:37.320139+0000","last_unstale":"2024-04-12T19:51:38.540899+0000","last_undegraded":"2024-04-12T19:51:38.540899+0000","last_fullsized":"2024-04-12T19:51:38.540899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T21:40:58.067392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.598509+0000","last_change":"2024-04-12T19:51:36.323335+0000","last_active":"2024-04-12T19:51:39.598509+0000","last_peered":"2024-04-12T19:51:39.598509+0000","last_clean":"2024-04-12T19:51:39.598509+0000","last_became_active":"2024-04-12T19:51:36.323010+0000","last_became_peered":"2024-04-12T19:51:36.323010+0000","last_unstale":"2024-04-12T19:51:39.598509+0000","last_undegraded":"2024-04-12T19:51:39.598509+0000","last_fullsized":"2024-04-12T19:51:39.598509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:15:41.032489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540055+0000","last_change":"2024-04-12T19:51:37.327527+0000","last_active":"2024-04-12T19:51:38.540055+0000","last_peered":"2024-04-12T19:51:38.540055+0000","last_clean":"2024-04-12T19:51:38.540055+0000","last_became_active":"2024-04-12T19:51:37.327226+0000","last_became_peered":"2024-04-12T19:51:37.327226+0000","last_unstale":"2024-04-12T19:51:38.540055+0000","last_undegraded":"2024-04-12T19:51:38.540055+0000","last_fullsized":"2024-04-12T19:51:38.540055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:44:30.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641881+0000","last_change":"2024-04-12T19:51:36.315742+0000","last_active":"2024-04-12T19:51:39.641881+0000","last_peered":"2024-04-12T19:51:39.641881+0000","last_clean":"2024-04-12T19:51:39.641881+0000","last_became_active":"2024-04-12T19:51:36.315459+0000","last_became_peered":"2024-04-12T19:51:36.315459+0000","last_unstale":"2024-04-12T19:51:39.641881+0000","last_undegraded":"2024-04-12T19:51:39.641881+0000","last_fullsized":"2024-04-12T19:51:39.641881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:58:44.538168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.321585+0000","last_change":"2024-04-12T19:51:37.322210+0000","last_active":"2024-04-12T19:51:38.321585+0000","last_peered":"2024-04-12T19:51:38.321585+0000","last_clean":"2024-04-12T19:51:38.321585+0000","last_became_active":"2024-04-12T19:51:37.321271+0000","last_became_peered":"2024-04-12T19:51:37.321271+0000","last_unstale":"2024-04-12T19:51:38.321585+0000","last_undegraded":"2024-04-12T19:51:38.321585+0000","last_fullsized":"2024-04-12T19:51:38.321585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T02:21:58.716784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.540066+0000","last_change":"2024-04-12T19:51:36.324603+0000","last_active":"2024-04-12T19:51:38.540066+0000","last_peered":"2024-04-12T19:51:38.540066+0000","last_clean":"2024-04-12T19:51:38.540066+0000","last_became_active":"2024-04-12T19:51:36.322196+0000","last_became_peered":"2024-04-12T19:51:36.322196+0000","last_unstale":"2024-04-12T19:51:38.540066+0000","last_undegraded":"2024-04-12T19:51:38.540066+0000","last_fullsized":"2024-04-12T19:51:38.540066+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:58:56.144991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.608151+0000","last_change":"2024-04-12T19:51:37.319380+0000","last_active":"2024-04-12T19:51:39.608151+0000","last_peered":"2024-04-12T19:51:39.608151+0000","last_clean":"2024-04-12T19:51:39.608151+0000","last_became_active":"2024-04-12T19:51:37.317927+0000","last_became_peered":"2024-04-12T19:51:37.317927+0000","last_unstale":"2024-04-12T19:51:39.608151+0000","last_undegraded":"2024-04-12T19:51:39.608151+0000","last_fullsized":"2024-04-12T19:51:39.608151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:36.272283+0000","last_clean_scrub_stamp":"2024-04-12T19:51:36.272283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T19:53:25.390782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:38.322279+0000","last_change":"2024-04-12T19:51:36.320886+0000","last_active":"2024-04-12T19:51:38.322279+0000","last_peered":"2024-04-12T19:51:38.322279+0000","last_clean":"2024-04-12T19:51:38.322279+0000","last_became_active":"2024-04-12T19:51:36.320612+0000","last_became_peered":"2024-04-12T19:51:36.320612+0000","last_unstale":"2024-04-12T19:51:38.322279+0000","last_undegraded":"2024-04-12T19:51:38.322279+0000","last_fullsized":"2024-04-12T19:51:38.322279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:51:35.255785+0000","last_clean_scrub_stamp":"2024-04-12T19:51:35.255785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T07:38:23.890393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":192,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-12T19:51:39.641115+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:51:39.641115+0000","last_peered":"2024-04-12T19:51:39.641115+0000","last_clean":"2024-04-12T19:51:39.641115+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:51:39.641115+0000","last_undegraded":"2024-04-12T19:51:39.641115+0000","last_fullsized":"2024-04-12T19:51:39.641115+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":742422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":153106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":152248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":741910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:51:47.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-12T19:51:47.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-12T19:51:47.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.574+0000 7fa882200640 1 Processor -- start 2024-04-12T19:51:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.574+0000 7fa882200640 1 -- start start 2024-04-12T19:51:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa882200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa87c15a440 0x7fa87c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa882200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c15d520 con 0x7fa87c06fc80 2024-04-12T19:51:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa882200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87c15d800 con 0x7fa87c072150 2024-04-12T19:51:47.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa882200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa87c06e280 con 0x7fa87c15a440 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa87b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa87c15a440 0x7fa87c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa87aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa87c06fc80 0x7fa87c0cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53926/0 (socket says 172.21.15.32:53926) 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.576+0000 7fa87aa00640 1 -- 172.21.15.32:0/3691253004 learned_addr learned my addr 172.21.15.32:0/3691253004 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.577+0000 7fa87b400640 1 -- 172.21.15.32:0/3691253004 >> v1:172.21.15.169:6789/0 conn(0x7fa87c072150 legacy=0x7fa87c06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.577+0000 7fa87b400640 1 -- 172.21.15.32:0/3691253004 >> v1:172.21.15.32:6789/0 conn(0x7fa87c06fc80 legacy=0x7fa87c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.577+0000 7fa87b400640 1 -- 172.21.15.32:0/3691253004 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c15dae0 con 0x7fa87c15a440 2024-04-12T19:51:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.578+0000 7fa87b400640 1 --2- 172.21.15.32:0/3691253004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c15a440 0x7fa87c15cef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa87c06e400 tx=0x7fa86c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6b37bed488af44 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.579+0000 7fa87a000640 1 -- 172.21.15.32:0/3691253004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c035020 con 0x7fa87c15a440 2024-04-12T19:51:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.579+0000 7fa87a000640 1 -- 172.21.15.32:0/3691253004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c031830 con 0x7fa87c15a440 2024-04-12T19:51:47.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.579+0000 7fa87a000640 1 -- 172.21.15.32:0/3691253004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c031c00 con 0x7fa87c15a440 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.579+0000 7fa882200640 1 -- 172.21.15.32:0/3691253004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c15a440 msgr2=0x7fa87c15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.579+0000 7fa882200640 1 --2- 172.21.15.32:0/3691253004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c15a440 0x7fa87c15cef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa87c06e400 tx=0x7fa86c02b8d0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.580+0000 7fa882200640 1 -- 172.21.15.32:0/3691253004 shutdown_connections 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.580+0000 7fa882200640 1 --2- 172.21.15.32:0/3691253004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c15a440 0x7fa87c15cef0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.580+0000 7fa882200640 1 -- 172.21.15.32:0/3691253004 >> 172.21.15.32:0/3691253004 conn(0x7fa87c0b2460 msgr2=0x7fa87c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.580+0000 7fa882200640 1 -- 172.21.15.32:0/3691253004 shutdown_connections 2024-04-12T19:51:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.580+0000 7fa882200640 1 -- 172.21.15.32:0/3691253004 wait complete. 2024-04-12T19:51:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.582+0000 7fa882200640 1 Processor -- start 2024-04-12T19:51:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.582+0000 7fa882200640 1 -- start start 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 0x7fa87c159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 0x7fa87c141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c15ddc0 con 0x7fa87c06fc80 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c15a820 con 0x7fa87c15a440 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa882200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa87c06f510 con 0x7fa87c072150 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa87b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa87b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58234/0 (socket says 172.21.15.32:58234) 2024-04-12T19:51:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.583+0000 7fa87b400640 1 -- 172.21.15.32:0/1157369706 learned_addr learned my addr 172.21.15.32:0/1157369706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87aa00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 0x7fa87c159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 -- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 msgr2=0x7fa87c159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87be00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 0x7fa87c141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 0x7fa87c159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 -- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 msgr2=0x7fa87c141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 0x7fa87c141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 -- 172.21.15.32:0/1157369706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c143280 con 0x7fa87c06fc80 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87aa00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 0x7fa87c159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87be00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 0x7fa87c141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa87b400640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009890 tx=0x7fa86c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c03a9b0 con 0x7fa87c06fc80 2024-04-12T19:51:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.584+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c03ab50 con 0x7fa87c06fc80 2024-04-12T19:51:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.585+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0443f0 con 0x7fa87c06fc80 2024-04-12T19:51:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.585+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c143810 con 0x7fa87c06fc80 2024-04-12T19:51:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.585+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c143dd0 con 0x7fa87c06fc80 2024-04-12T19:51:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.586+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c068c80 con 0x7fa87c06fc80 2024-04-12T19:51:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.587+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa86c038020 con 0x7fa87c06fc80 2024-04-12T19:51:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.588+0000 7fa873400640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 0x7fa85c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.588+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa86c0ad910 con 0x7fa87c06fc80 2024-04-12T19:51:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.588+0000 7fa87aa00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 0x7fa85c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.589+0000 7fa87aa00640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 0x7fa85c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa87c15ecb0 tx=0x7fa868009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.593+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa86c03f070 con 0x7fa87c06fc80 2024-04-12T19:51:47.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.878+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa87c143fa0 con 0x7fa87c06fc80 2024-04-12T19:51:47.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.884+0000 7fa873400640 1 -- 172.21.15.32:0/1157369706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fa86c07bca0 con 0x7fa87c06fc80 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA6kBlmq9xnMxAA2e+ydW8KXvsNYpAvO4M4Xg== 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-12T19:51:47.883 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-12T19:51:47.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.886+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 msgr2=0x7fa85c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.886+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 0x7fa85c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa87c15ecb0 tx=0x7fa868009040 comp rx=0 tx=0).stop 2024-04-12T19:51:47.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.886+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 msgr2=0x7fa87c140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:47.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.886+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009890 tx=0x7fa86c0046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 shutdown_connections 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa85c06c8f0 0x7fa85c06edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa87c15a440 0x7fa87c141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa87c072150 0x7fa87c159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 --2- 172.21.15.32:0/1157369706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c06fc80 0x7fa87c140340 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 >> 172.21.15.32:0/1157369706 conn(0x7fa87c0b2460 msgr2=0x7fa87c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 shutdown_connections 2024-04-12T19:51:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:47.887+0000 7fa882200640 1 -- 172.21.15.32:0/1157369706 wait complete. 2024-04-12T19:51:47.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.133+0000 7f6e4da00640 1 Processor -- start 2024-04-12T19:51:48.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.134+0000 7f6e4da00640 1 -- start start 2024-04-12T19:51:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e48162020 0x7f6e48162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e480cf110 0x7f6e480d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e480d1ac0 0x7f6e480b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6e480d1f50 con 0x7f6e480cf110 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6e48163440 con 0x7f6e48162020 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e4da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480b7b30 con 0x7f6e480d1ac0 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.135+0000 7f6e3fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e480cf110 0x7f6e480d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e480cf110 0x7f6e480d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58246/0 (socket says 172.21.15.32:58246) 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 -- 172.21.15.32:0/2500802000 learned_addr learned my addr 172.21.15.32:0/2500802000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 -- 172.21.15.32:0/2500802000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e480d1ac0 msgr2=0x7f6e480b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/2500802000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e480d1ac0 0x7f6e480b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 -- 172.21.15.32:0/2500802000 >> v2:172.21.15.169:3300/0 conn(0x7f6e48162020 msgr2=0x7f6e48162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/2500802000 >> v2:172.21.15.169:3300/0 conn(0x7f6e48162020 0x7f6e48162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.136+0000 7f6e3fe00640 1 -- 172.21.15.32:0/2500802000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480d0090 con 0x7f6e480cf110 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.137+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/2500802000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e480d1580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e4808cd90 tx=0x7f6e3002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ccd8532de953e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.137+0000 7f6e47400640 1 -- 172.21.15.32:0/2500802000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30035020 con 0x7f6e480cf110 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.137+0000 7f6e47400640 1 -- 172.21.15.32:0/2500802000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e30031880 con 0x7f6e480cf110 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.137+0000 7f6e47400640 1 -- 172.21.15.32:0/2500802000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30031c50 con 0x7f6e480cf110 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 -- 172.21.15.32:0/2500802000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 msgr2=0x7f6e480d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 --2- 172.21.15.32:0/2500802000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e480d1580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e4808cd90 tx=0x7f6e3002b920 comp rx=0 tx=0).stop 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 -- 172.21.15.32:0/2500802000 shutdown_connections 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 --2- 172.21.15.32:0/2500802000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e480d1580 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 --2- 172.21.15.32:0/2500802000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e480d1ac0 0x7f6e480b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 --2- 172.21.15.32:0/2500802000 >> v2:172.21.15.169:3300/0 conn(0x7f6e48162020 0x7f6e48162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 -- 172.21.15.32:0/2500802000 >> 172.21.15.32:0/2500802000 conn(0x7f6e4806cf00 msgr2=0x7f6e4806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 -- 172.21.15.32:0/2500802000 shutdown_connections 2024-04-12T19:51:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.138+0000 7f6e4da00640 1 -- 172.21.15.32:0/2500802000 wait complete. 2024-04-12T19:51:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.140+0000 7f6e4da00640 1 Processor -- start 2024-04-12T19:51:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.140+0000 7f6e4da00640 1 -- start start 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e480d1ac0 0x7f6e480cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e48162020 0x7f6e481f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480cfdb0 con 0x7f6e480cf110 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480cf880 con 0x7f6e480d1ac0 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.141+0000 7f6e4da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480d2b90 con 0x7f6e48162020 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58260/0 (socket says 172.21.15.32:58260) 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 -- 172.21.15.32:0/810694361 learned_addr learned my addr 172.21.15.32:0/810694361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e480d1ac0 0x7f6e480cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e4d000640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e48162020 0x7f6e481f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 -- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e48162020 msgr2=0x7f6e481f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e48162020 0x7f6e481f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 -- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e480d1ac0 msgr2=0x7f6e480cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e480d1ac0 0x7f6e480cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.142+0000 7f6e47e00640 1 -- 172.21.15.32:0/810694361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e481f55c0 con 0x7f6e480cf110 2024-04-12T19:51:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e47e00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800b4a0 tx=0x7f6e3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38014070 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e38010610 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38010a00 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e481f58a0 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.143+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e480c8c00 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.144+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e480755c0 con 0x7f6e480cf110 2024-04-12T19:51:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.145+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e38017070 con 0x7f6e480cf110 2024-04-12T19:51:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.146+0000 7f6e44c00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.146+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6e3808b600 con 0x7f6e480cf110 2024-04-12T19:51:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.147+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.148+0000 7f6e3fe00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 0x7f6e2406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e30036000 tx=0x7f6e3003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.153+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e38059a20 con 0x7f6e480cf110 2024-04-12T19:51:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.398+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e480cd540 con 0x7f6e480cf110 2024-04-12T19:51:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.398+0000 7f6e44c00640 1 -- 172.21.15.32:0/810694361 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f6e38059320 con 0x7f6e480cf110 2024-04-12T19:51:48.399 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:48.399 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:51:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 msgr2=0x7f6e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 0x7f6e2406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e30036000 tx=0x7f6e3003c040 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 msgr2=0x7f6e48151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800b4a0 tx=0x7f6e3800b960 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 shutdown_connections 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6e2406c8f0 0x7f6e2406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e48162020 0x7f6e481f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e480d1ac0 0x7f6e480cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 --2- 172.21.15.32:0/810694361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e480cf110 0x7f6e48151c50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 >> 172.21.15.32:0/810694361 conn(0x7f6e4806cf00 msgr2=0x7f6e4815f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:48.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 shutdown_connections 2024-04-12T19:51:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:48.401+0000 7f6e4da00640 1 -- 172.21.15.32:0/810694361 wait complete. 2024-04-12T19:51:48.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-12T19:51:36.786433+0000', 'modified': '2024-04-12T19:51:39.843794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5022': {'gid': 5022, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6836/1336962272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1336962272}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 1336962272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5022, 'qdb_cluster': [5022]} 2024-04-12T19:51:48.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-12T19:51:48.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-12T19:51:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.053+0000 7f0fd1400640 1 Processor -- start 2024-04-12T19:51:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.053+0000 7f0fd1400640 1 -- start start 2024-04-12T19:51:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fd1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fcc0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fd1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fcc0b59b0 0x7f0fcc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fd1400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fcc0cd940 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fd1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0fcc0ce550 con 0x7f0fcc0b59b0 2024-04-12T19:51:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fd1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cdc20 con 0x7f0fcc15d770 2024-04-12T19:51:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fcbe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fcc0b0cb0 0x7f0fcc0ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53928/0 (socket says 172.21.15.32:53928) 2024-04-12T19:51:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.055+0000 7f0fcbe00640 1 -- 172.21.15.32:0/3711839041 learned_addr learned my addr 172.21.15.32:0/3711839041 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508452047 0 0) 0x7f0fcc0cd940 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fb0003520 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/3711839041 >> v2:172.21.15.169:3300/0 conn(0x7f0fcc0b59b0 0x7f0fcc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fcc0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4165730771 0 0) 0x7f0fb0003520 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 msgr2=0x7f0fcc0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fcc0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 >> v2:172.21.15.169:3300/0 conn(0x7f0fcc0b59b0 msgr2=0x7f0fcc0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 --2- 172.21.15.32:0/3711839041 >> v2:172.21.15.169:3300/0 conn(0x7f0fcc0b59b0 0x7f0fcc0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc0cdf00 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f0fbc001670 con 0x7f0fcc0b0cb0 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 >> v1:172.21.15.32:6789/0 conn(0x7f0fcc0b0cb0 legacy=0x7f0fcc0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb00039b0 0x7f0fb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0004800 0x7f0fb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0ce550 con 0x7f0fb0004800 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cdc20 con 0x7f0fb00039b0 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.056+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb000bac0 con 0x7f0fcc15d770 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0004800 0x7f0fb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb00039b0 0x7f0fb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 -- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 msgr2=0x7f0fb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc15d770 0x7f0fb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 -- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb00039b0 msgr2=0x7f0fb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb00039b0 0x7f0fb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.057+0000 7f0fc3e00640 1 -- 172.21.15.32:0/3711839041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb0003520 con 0x7f0fb0004800 2024-04-12T19:51:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.058+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0004800 0x7f0fb0004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400ba00 tx=0x7f0fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4279c569873957c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4036020 con 0x7f0fb0004800 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb40325f0 con 0x7f0fb0004800 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fcb400640 1 -- 172.21.15.32:0/3711839041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400b600 con 0x7f0fb0004800 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fd1400640 1 -- 172.21.15.32:0/3711839041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0004800 msgr2=0x7f0fb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fd1400640 1 --2- 172.21.15.32:0/3711839041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb0004800 0x7f0fb0004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400ba00 tx=0x7f0fb0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.059+0000 7f0fd0a00640 1 -- 172.21.15.32:0/3711839041 reap_dead start 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.060+0000 7f0fd1400640 1 -- 172.21.15.32:0/3711839041 shutdown_connections 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.060+0000 7f0fd1400640 1 -- 172.21.15.32:0/3711839041 >> 172.21.15.32:0/3711839041 conn(0x7f0fcc0b23a0 msgr2=0x7f0fcc0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.060+0000 7f0fd1400640 1 -- 172.21.15.32:0/3711839041 shutdown_connections 2024-04-12T19:51:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.060+0000 7f0fd1400640 1 -- 172.21.15.32:0/3711839041 wait complete. 2024-04-12T19:51:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.061+0000 7f0fd1400640 1 Processor -- start 2024-04-12T19:51:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.062+0000 7f0fd1400640 1 -- start start 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.062+0000 7f0fd1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fcc1512e0 0x7f0fcc0af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.062+0000 7f0fd1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc0af8b0 0x7f0fcc15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd1400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cd020 con 0x7f0fcc15f740 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0b1090 con 0x7f0fcc1512e0 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc0cd1a0 con 0x7f0fcc0af8b0 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58276/0 (socket says 172.21.15.32:58276) 2024-04-12T19:51:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 -- 172.21.15.32:0/4246168354 learned_addr learned my addr 172.21.15.32:0/4246168354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc0af8b0 0x7f0fcc15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 -- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc0af8b0 msgr2=0x7f0fcc15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fcc1512e0 0x7f0fcc0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc0af8b0 0x7f0fcc15f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 -- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fcc1512e0 msgr2=0x7f0fcc0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fcc1512e0 0x7f0fcc0af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.063+0000 7f0fd0a00640 1 -- 172.21.15.32:0/4246168354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc0b3840 con 0x7f0fcc15f740 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0b1a10 tx=0x7f0fc4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fc8c00640 1 -- 172.21.15.32:0/4246168354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4014410 con 0x7f0fcc15f740 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fc8c00640 1 -- 172.21.15.32:0/4246168354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc4019070 con 0x7f0fcc15f740 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fc8c00640 1 -- 172.21.15.32:0/4246168354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4014810 con 0x7f0fcc15f740 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc0b3ab0 con 0x7f0fcc15f740 2024-04-12T19:51:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.064+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc1516c0 con 0x7f0fcc15f740 2024-04-12T19:51:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.066+0000 7f0fc8c00640 1 -- 172.21.15.32:0/4246168354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fc40149b0 con 0x7f0fcc15f740 2024-04-12T19:51:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.067+0000 7f0fc8c00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.067+0000 7f0fc8c00640 1 -- 172.21.15.32:0/4246168354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0fc408c170 con 0x7f0fcc15f740 2024-04-12T19:51:49.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.067+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.068+0000 7f0fd57f2740 1 Processor -- start 2024-04-12T19:51:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.068+0000 7f0fd57f2740 1 -- start start 2024-04-12T19:51:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.068+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 0x7f0fa806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc001bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156a49d0 0x5556156a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555615609990 con 0x5556156cbb90 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556155b0530 con 0x5556156cb7b0 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x555615605950 con 0x5556156a49d0 2024-04-12T19:51:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58292/0 (socket says 172.21.15.32:58292) 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.069+0000 7f0fc3e00640 1 -- 172.21.15.32:0/1521978069 learned_addr learned my addr 172.21.15.32:0/1521978069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156a49d0 0x5556156a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 -- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156a49d0 msgr2=0x5556156a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156a49d0 0x5556156a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 -- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 msgr2=0x5556156ea8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156ea8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 -- 172.21.15.32:0/1521978069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561531c2d0 con 0x5556156cbb90 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.070+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fb402bf40 tx=0x7f0fb4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200cebd8606c589c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fbbe00640 1 -- 172.21.15.32:0/1521978069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4036040 con 0x5556156cbb90 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fbbe00640 1 -- 172.21.15.32:0/1521978069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb403b420 con 0x5556156cbb90 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fbbe00640 1 -- 172.21.15.32:0/1521978069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb403b7f0 con 0x5556156cbb90 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 -- 172.21.15.32:0/1521978069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 msgr2=0x5556156a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fb402bf40 tx=0x7f0fb4032000 comp rx=0 tx=0).stop 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 -- 172.21.15.32:0/1521978069 shutdown_connections 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156a49d0 0x5556156a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a4490 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/1521978069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 -- 172.21.15.32:0/1521978069 >> 172.21.15.32:0/1521978069 conn(0x5556156a0a70 msgr2=0x5556156e95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 -- 172.21.15.32:0/1521978069 shutdown_connections 2024-04-12T19:51:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.071+0000 7f0fd57f2740 1 -- 172.21.15.32:0/1521978069 wait complete. 2024-04-12T19:51:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.072+0000 7f0fd57f2740 1 -- start start 2024-04-12T19:51:49.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.072+0000 7f0fd57f2740 1 Processor -- start 2024-04-12T19:51:49.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.073+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.073+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cbb90 0x5556156e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.073+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156e45d0 0x5556156e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55561569e8a0 con 0x5556156cb7b0 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556155c38f0 con 0x5556156cbb90 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5556156a4e50 con 0x5556156e45d0 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fd57f2740 10 client.0 _ll_register_callbacks cb 0x5556155c32e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58298/0 (socket says 172.21.15.32:58298) 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 -- 172.21.15.32:0/3102929606 learned_addr learned my addr 172.21.15.32:0/3102929606 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156e45d0 0x5556156e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 -- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156e45d0 msgr2=0x5556156e4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156e45d0 0x5556156e4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 -- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cbb90 msgr2=0x5556156e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cbb90 0x5556156e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.074+0000 7f0fcbe00640 1 -- 172.21.15.32:0/3102929606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556153e66b0 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00ffe0 tx=0x7f0fbc009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3474b2d2e31e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc010040 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0fbc022a80 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb403b420 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0fbc022c80 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fd57f2740 10 client.5211 fetch_fsmap learned FSMap version 16 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.075+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5556156c5c40 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb403b420 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f0fbc013070 con 0x5556156cb7b0 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 10 client.5211 fetch_fsmap finished waiting for FSMap version 16 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 10 client.5211 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-12T19:51:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 20 client.5211 populate_metadata read hostname 'smithi032' 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156ecaf0 0x555615707300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 4 client.5211 mds_command: new command op to 5022 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 --> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556155afb40 con 0x5556156ecaf0 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.076+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156ecaf0 0x555615707300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.077+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156ecaf0 0x555615707300 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c0659185988547 server_cookie=31a033335284a93 in_seq=0 out_seq=0 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.077+0000 7f0fb8c00640 10 client.5211 ms_handle_connect on v2:172.21.15.32:6835/1336962272 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fb8c00640 1 -- 172.21.15.32:0/3102929606 <== mds.0 v2:172.21.15.32:6835/1336962272 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556155afb40 con 0x5556156ecaf0 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fb8c00640 10 client.5211 handle_command_reply: tid=0 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 client.5211 shutdown 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 msgr2=0x5556156e59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00ffe0 tx=0x7f0fbc009070 comp rx=0 tx=0).stop 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 shutdown_connections 2024-04-12T19:51:49.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156e45d0 0x5556156e4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156ecaf0 0x555615707300 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cbb90 0x5556156e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/3102929606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cb7b0 0x5556156e59a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 >> 172.21.15.32:0/3102929606 conn(0x5556156a49d0 msgr2=0x5556156c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fb8c00640 0 client.5211 ms_handle_reset on v2:172.21.15.32:6835/1336962272 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 shutdown_connections 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 1 -- 172.21.15.32:0/3102929606 wait complete. 2024-04-12T19:51:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.078+0000 7f0fd57f2740 20 client.5211 trim_cache size 0 max 16384 2024-04-12T19:51:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.098+0000 7f0fd57f2740 1 Processor -- start 2024-04-12T19:51:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.098+0000 7f0fd57f2740 1 -- start start 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cbb90 0x55561570ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc200a70 con 0x5556156e45d0 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc1f59b0 con 0x5556156cb7b0 2024-04-12T19:51:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc200480 con 0x5556156cbb90 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.099+0000 7f0fd0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58304/0 (socket says 172.21.15.32:58304) 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 -- 172.21.15.32:0/2960261034 learned_addr learned my addr 172.21.15.32:0/2960261034 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cbb90 0x55561570ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 -- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cbb90 msgr2=0x55561570ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cbb90 0x55561570ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 -- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 msgr2=0x5556156eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 -- 172.21.15.32:0/2960261034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556156e4dc0 con 0x5556156e45d0 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0fc401d000 tx=0x7f0fc4059a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a12218ff0a12df server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fbb400640 1 -- 172.21.15.32:0/2960261034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4072070 con 0x5556156e45d0 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fbb400640 1 -- 172.21.15.32:0/2960261034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc405e240 con 0x5556156e45d0 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.100+0000 7f0fbb400640 1 -- 172.21.15.32:0/2960261034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4062a70 con 0x5556156e45d0 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 -- 172.21.15.32:0/2960261034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 msgr2=0x555615759f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0fc401d000 tx=0x7f0fc4059a90 comp rx=0 tx=0).stop 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 -- 172.21.15.32:0/2960261034 shutdown_connections 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156e45d0 0x555615759f50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cbb90 0x55561570ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/2960261034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156cb7b0 0x5556156eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 -- 172.21.15.32:0/2960261034 >> 172.21.15.32:0/2960261034 conn(0x5556156a49d0 msgr2=0x55561569e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 -- 172.21.15.32:0/2960261034 shutdown_connections 2024-04-12T19:51:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.101+0000 7f0fd57f2740 1 -- 172.21.15.32:0/2960261034 wait complete. 2024-04-12T19:51:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.102+0000 7f0fd57f2740 1 -- start start 2024-04-12T19:51:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.102+0000 7f0fd57f2740 1 Processor -- start 2024-04-12T19:51:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cb7b0 0x555615756390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156e45d0 0x5556156a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5556156ec970 con 0x5556156cbb90 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556156ec4d0 con 0x5556156e45d0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5556156ecf40 con 0x5556156cb7b0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fd57f2740 10 client.0 _ll_register_callbacks cb 0x7f0fbc00fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58320/0 (socket says 172.21.15.32:58320) 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.103+0000 7f0fc3e00640 1 -- 172.21.15.32:0/761392316 learned_addr learned my addr 172.21.15.32:0/761392316 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cb7b0 0x555615756390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 -- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cb7b0 msgr2=0x555615756390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fd0a00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156e45d0 0x5556156a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cb7b0 0x555615756390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 -- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156e45d0 msgr2=0x5556156a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156e45d0 0x5556156a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 -- 172.21.15.32:0/761392316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556156eb4a0 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fc3e00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006fd0 tx=0x7f0fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda57b88a00ff263 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4036040 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb4004020 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.104+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fc405e240 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0fb400e000 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 10 client.5217 fetch_fsmap learned FSMap version 16 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5556156eaca0 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc405e240 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f0fb4004220 con 0x5556156cbb90 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 10 client.5217 fetch_fsmap finished waiting for FSMap version 16 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 10 client.5217 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 20 client.5217 populate_metadata read hostname 'smithi032' 2024-04-12T19:51:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156e8300 0x55561569d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 4 client.5217 mds_command: new command op to 5022 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-12T19:51:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 --> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] -- command(tid 0: {"prefix": "status"}) -- 0x5556156173b0 con 0x5556156e8300 2024-04-12T19:51:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.105+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156e8300 0x55561569d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fcbe00640 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156e8300 0x55561569d6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5fbd0c6b6a39a82 server_cookie=f801f6ba31d691f1 in_seq=0 out_seq=0 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fb8c00640 10 client.5217 ms_handle_connect on v2:172.21.15.32:6835/1336962272 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fb8c00640 1 -- 172.21.15.32:0/761392316 <== mds.0 v2:172.21.15.32:6835/1336962272 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5556156173b0 con 0x5556156e8300 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fb8c00640 10 client.5217 handle_command_reply: tid=0 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 client.5217 shutdown 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 msgr2=0x5556156a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006fd0 tx=0x7f0fb4004730 comp rx=0 tx=0).stop 2024-04-12T19:51:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 shutdown_connections 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:6835/1336962272,v1:172.21.15.32:6836/1336962272] conn(0x5556156e8300 0x55561569d6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556156e45d0 0x5556156a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5556156cbb90 0x5556156a2360 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fb8c00640 0 client.5217 ms_handle_reset on v2:172.21.15.32:6835/1336962272 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.106+0000 7f0fd57f2740 1 --2- 172.21.15.32:0/761392316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556156cb7b0 0x555615756390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 >> 172.21.15.32:0/761392316 conn(0x5556156a49d0 msgr2=0x5556156e9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 shutdown_connections 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd57f2740 1 -- 172.21.15.32:0/761392316 wait complete. 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd57f2740 20 client.5217 trim_cache size 0 max 16384 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:51:49.110 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "47426077-081d-47b8-bfde-ce036e9b4cf8", 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5022, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 11.797593789, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 16, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 35, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 35, 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 20.768382762000002 2024-04-12T19:51:49.111 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 msgr2=0x7f0fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 0x7f0fa806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc001bc0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 msgr2=0x7f0fcc0ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.107+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0b1a10 tx=0x7f0fc4007700 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 shutdown_connections 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc15f740 0x7f0fcc0ad920 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fcc0af8b0 0x7f0fcc15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 --2- 172.21.15.32:0/4246168354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fcc1512e0 0x7f0fcc0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 >> 172.21.15.32:0/4246168354 conn(0x7f0fcc0b23a0 msgr2=0x7f0fcc15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 shutdown_connections 2024-04-12T19:51:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.108+0000 7f0fd1400640 1 -- 172.21.15.32:0/4246168354 wait complete. 2024-04-12T19:51:49.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-12T19:51:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.331+0000 7f9122200640 1 Processor -- start 2024-04-12T19:51:49.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.331+0000 7f9122200640 1 -- start start 2024-04-12T19:51:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.332+0000 7f9122200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f911c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.332+0000 7f9122200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f911c0b3470 0x7f911c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.332+0000 7f9122200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911c0d2d90 con 0x7f911c15eac0 2024-04-12T19:51:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.332+0000 7f9122200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f911c0d1cd0 con 0x7f911c0b3470 2024-04-12T19:51:49.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.332+0000 7f9122200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d3070 con 0x7f911c0d1300 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.333+0000 7f911b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f911c15eac0 0x7f911c0d3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53938/0 (socket says 172.21.15.32:53938) 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.333+0000 7f911b400640 1 -- 172.21.15.32:0/4135747435 learned_addr learned my addr 172.21.15.32:0/4135747435 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.333+0000 7f911be00640 1 --2- 172.21.15.32:0/4135747435 >> v2:172.21.15.169:3300/0 conn(0x7f911c0b3470 0x7f911c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.333+0000 7f911aa00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f911c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.333+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944015159 0 0) 0x7f911c0d2d90 con 0x7f911c15eac0 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9100003520 con 0x7f911c15eac0 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3012112413 0 0) 0x7f9100003520 con 0x7f911c15eac0 2024-04-12T19:51:49.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 msgr2=0x7f911c0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f911c0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 >> v2:172.21.15.169:3300/0 conn(0x7f911c0b3470 msgr2=0x7f911c0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 --2- 172.21.15.32:0/4135747435 >> v2:172.21.15.169:3300/0 conn(0x7f911c0b3470 0x7f911c0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0d3350 con 0x7f911c15eac0 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f910c001670 con 0x7f911c15eac0 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 >> v1:172.21.15.32:6789/0 conn(0x7f911c15eac0 legacy=0x7f911c0d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f91000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100003f80 0x7f9100004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9100004d40 0x7f91000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d1cd0 con 0x7f9100004d40 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d3070 con 0x7f9100003f80 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.334+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f910000b9c0 con 0x7f911c0d1300 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9100004d40 0x7f91000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911b400640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f91000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911aa00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100003f80 0x7f9100004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 -- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 msgr2=0x7f91000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0d1300 0x7f91000039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 -- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100003f80 msgr2=0x7f9100004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9100003f80 0x7f9100004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.335+0000 7f911be00640 1 -- 172.21.15.32:0/4135747435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9100003520 con 0x7f9100004d40 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.336+0000 7f911be00640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9100004d40 0x7f91000051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f911400e170 tx=0x7f9100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c786566e6b12e498 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.336+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114039070 con 0x7f9100004d40 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.336+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911402cc20 con 0x7f9100004d40 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.336+0000 7f911a000640 1 -- 172.21.15.32:0/4135747435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114009c10 con 0x7f9100004d40 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 -- 172.21.15.32:0/4135747435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9100004d40 msgr2=0x7f91000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 --2- 172.21.15.32:0/4135747435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9100004d40 0x7f91000051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f911400e170 tx=0x7f9100005830 comp rx=0 tx=0).stop 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f911be00640 1 -- 172.21.15.32:0/4135747435 reap_dead start 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 -- 172.21.15.32:0/4135747435 shutdown_connections 2024-04-12T19:51:49.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 -- 172.21.15.32:0/4135747435 >> 172.21.15.32:0/4135747435 conn(0x7f911c073690 msgr2=0x7f911c06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 -- 172.21.15.32:0/4135747435 shutdown_connections 2024-04-12T19:51:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.337+0000 7f9122200640 1 -- 172.21.15.32:0/4135747435 wait complete. 2024-04-12T19:51:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.339+0000 7f9122200640 1 Processor -- start 2024-04-12T19:51:49.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.339+0000 7f9122200640 1 -- start start 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0b60f0 0x7f911c1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c161680 0x7f911c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f911c15eea0 con 0x7f911c1612a0 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0d3910 con 0x7f911c161680 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f9122200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0d4590 con 0x7f911c0b60f0 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f911b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f911b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58332/0 (socket says 172.21.15.32:58332) 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.340+0000 7f911b400640 1 -- 172.21.15.32:0/2106924516 learned_addr learned my addr 172.21.15.32:0/2106924516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911aa00640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0b60f0 0x7f911c1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 -- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0b60f0 msgr2=0x7f911c1f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0b60f0 0x7f911c1f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 -- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c161680 msgr2=0x7f911c1f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c161680 0x7f911c1f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 -- 172.21.15.32:0/2106924516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c0ce990 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f911b400640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f910c00dae0 tx=0x7f910c001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.341+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c019070 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.342+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910c004cb0 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.342+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c004e60 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.342+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c1f4590 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.342+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1f4b50 con 0x7f911c1612a0 2024-04-12T19:51:49.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.343+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c068c80 con 0x7f911c1612a0 2024-04-12T19:51:49.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.344+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f910c0072d0 con 0x7f911c1612a0 2024-04-12T19:51:49.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.345+0000 7f9113400640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 0x7f90fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:49.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.345+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f910c08b590 con 0x7f911c1612a0 2024-04-12T19:51:49.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.345+0000 7f911aa00640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 0x7f90fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:49.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.346+0000 7f911aa00640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 0x7f90fc06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b9770 tx=0x7f9108008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:49.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.350+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f910c0599c0 con 0x7f911c1612a0 2024-04-12T19:51:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.600+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f911c0d2550 con 0x7f911c1612a0 2024-04-12T19:51:49.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.601+0000 7f9113400640 1 -- 172.21.15.32:0/2106924516 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f910c0592c0 con 0x7f911c1612a0 2024-04-12T19:51:49.601 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-12T19:51:49.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 msgr2=0x7f90fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 0x7f90fc06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b9770 tx=0x7f9108008040 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 msgr2=0x7f911c1f1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f910c00dae0 tx=0x7f910c001da0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 shutdown_connections 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f90fc06c8f0 0x7f90fc06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c161680 0x7f911c1f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0b60f0 0x7f911c1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 --2- 172.21.15.32:0/2106924516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c1612a0 0x7f911c1f1360 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 >> 172.21.15.32:0/2106924516 conn(0x7f911c073690 msgr2=0x7f911c0c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.603+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 shutdown_connections 2024-04-12T19:51:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49.604+0000 7f9122200640 1 -- 172.21.15.32:0/2106924516 wait complete. 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T19:51:49.622 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-12T19:51:49.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:49.623 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:51:49.676 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42472sec preferred_lft 42472sec 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T19:51:49.677 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:b0:da:ad:ee:1d brd ff:ff:ff:ff:ff:ff 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::a47c:8fff:feea:b793/64 scope link 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether a6:fb:7c:29:2e:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::a4fb:7cff:fe29:2eb6/64 scope link 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.678 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-12T19:51:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 72:b0:da:ad:ee:1d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-12T19:51:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::70b0:daff:fead:ee1d/64 scope link 2024-04-12T19:51:49.679 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-12T19:51:49.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:49.679 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T19:51:49.774 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-12T19:51:49.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:49.779 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-12T19:51:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-12T19:51:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T19:51:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-12T19:51:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-12T19:51:49.838 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-12T19:51:49.839 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-12T19:51:49.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T19:51:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:51:50.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.102 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:51:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-12T19:51:50.149 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-12T19:51:50.150 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-12T19:51:50.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-12T19:51:50.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:51:50.252 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-12T19:51:50.311 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-12T19:51:50.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T19:51:50.311 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-12T19:51:50.365 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-12T19:51:50.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:51:50.513 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:51:50.513 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:51:50.513 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:51:50.536 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:51:50.536 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:51:50.536 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:51:50.536 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:51:50.536 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:51:50.570 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-12T19:51:50.570 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790" 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2024-04-12T19:51:50.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47426077-081d-47b8-bfde-ce036e9b4cf8" will pass to kernel 2024-04-12T19:51:50.573 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-12T19:51:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-04-12T19:51:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-12T19:51:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-04-12T19:51:50.573 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-12T19:51:50.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-12T19:51:50.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-12T19:51:50.675 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-12T19:51:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-12T19:51:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:51:50.814 INFO:teuthology.orchestra.run.smithi032.stdout:client4688 2024-04-12T19:51:50.814 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:50.815 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:50.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.871 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir mydir) 2024-04-12T19:51:50.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:50.940 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch mydir/alpha) 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir").st_ino) 2024-04-12T19:51:51.002 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-12T19:51:51.148 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-12T19:51:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/mydir/alpha").st_ino) 2024-04-12T19:51:51.162 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-12T19:51:51.356 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-12T19:51:51.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T19:51:51.370 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:51.370 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:51.470 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:51.470 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:51.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.718+0000 7fc608200640 1 Processor -- start 2024-04-12T19:51:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.718+0000 7fc608200640 1 -- start start 2024-04-12T19:51:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.720+0000 7fc608200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.720+0000 7fc608200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 0x7fc60015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.720+0000 7fc608200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6000b4be0 con 0x7fc6001461f0 2024-04-12T19:51:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.720+0000 7fc608200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc6000a9640 con 0x7fc6000b3e10 2024-04-12T19:51:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.720+0000 7fc608200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000b4620 con 0x7fc6000cefc0 2024-04-12T19:51:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.721+0000 7fc604e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.721+0000 7fc606200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 0x7fc60015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.721+0000 7fc604e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:55250/0 (socket says 172.21.15.32:55250) 2024-04-12T19:51:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.721+0000 7fc604e00640 1 -- 172.21.15.32:0/3906138664 learned_addr learned my addr 172.21.15.32:0/3906138664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.721+0000 7fc605800640 1 --1- 172.21.15.32:0/3906138664 >> v1:172.21.15.169:6789/0 conn(0x7fc6001461f0 0x7fc6000ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:59538/0 (socket says 172.21.15.32:59538) 2024-04-12T19:51:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.722+0000 7fc604e00640 1 -- 172.21.15.32:0/3906138664 >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 msgr2=0x7fc60015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.722+0000 7fc604e00640 1 --2- 172.21.15.32:0/3906138664 >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 0x7fc60015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.722+0000 7fc604e00640 1 -- 172.21.15.32:0/3906138664 >> v1:172.21.15.169:6789/0 conn(0x7fc6001461f0 legacy=0x7fc6000ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.722+0000 7fc604e00640 1 -- 172.21.15.32:0/3906138664 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000b4900 con 0x7fc6000cefc0 2024-04-12T19:51:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.722+0000 7fc606200640 1 --2- 172.21.15.32:0/3906138664 >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 0x7fc60015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:51:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.723+0000 7fc604e00640 1 --2- 172.21.15.32:0/3906138664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0098a0 tx=0x7fc5ec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf5d2b593e48898 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.724+0000 7fc5ffe00640 1 -- 172.21.15.32:0/3906138664 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec034070 con 0x7fc6000cefc0 2024-04-12T19:51:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.724+0000 7fc5ffe00640 1 -- 172.21.15.32:0/3906138664 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ec030840 con 0x7fc6000cefc0 2024-04-12T19:51:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.724+0000 7fc5ffe00640 1 -- 172.21.15.32:0/3906138664 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec030c10 con 0x7fc6000cefc0 2024-04-12T19:51:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 -- 172.21.15.32:0/3906138664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 msgr2=0x7fc6000b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 --2- 172.21.15.32:0/3906138664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0098a0 tx=0x7fc5ec02b980 comp rx=0 tx=0).stop 2024-04-12T19:51:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 -- 172.21.15.32:0/3906138664 shutdown_connections 2024-04-12T19:51:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 --2- 172.21.15.32:0/3906138664 >> v2:172.21.15.169:3300/0 conn(0x7fc6000b3e10 0x7fc60015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 --2- 172.21.15.32:0/3906138664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000b38d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 -- 172.21.15.32:0/3906138664 >> 172.21.15.32:0/3906138664 conn(0x7fc6000b27b0 msgr2=0x7fc6000b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 -- 172.21.15.32:0/3906138664 shutdown_connections 2024-04-12T19:51:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.725+0000 7fc608200640 1 -- 172.21.15.32:0/3906138664 wait complete. 2024-04-12T19:51:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.727+0000 7fc608200640 1 Processor -- start 2024-04-12T19:51:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.727+0000 7fc608200640 1 -- start start 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6000b3e10 0x7fc6000c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001465d0 con 0x7fc6001461f0 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc60015d290 con 0x7fc6000b3e10 2024-04-12T19:51:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc608200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000b42a0 con 0x7fc6000cefc0 2024-04-12T19:51:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc606200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc606200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58340/0 (socket says 172.21.15.32:58340) 2024-04-12T19:51:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc606200640 1 -- 172.21.15.32:0/1038052091 learned_addr learned my addr 172.21.15.32:0/1038052091 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.729+0000 7fc605800640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6000b3e10 0x7fc6000c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc604e00640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 -- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 msgr2=0x7fc6000c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 -- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6000b3e10 msgr2=0x7fc6000c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6000b3e10 0x7fc6000c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 -- 172.21.15.32:0/1038052091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000c63b0 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc606200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800efc0 tx=0x7fc5f8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.730+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8018030 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.731+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f800cde0 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.731+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003bb0 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.731+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6000c6690 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.731+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000c6c50 con 0x7fc6001461f0 2024-04-12T19:51:51.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.732+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600068c80 con 0x7fc6001461f0 2024-04-12T19:51:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.733+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5f801b070 con 0x7fc6001461f0 2024-04-12T19:51:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.734+0000 7fc5fd600640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 0x7fc5e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.734+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f808c600 con 0x7fc6001461f0 2024-04-12T19:51:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.734+0000 7fc605800640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 0x7fc5e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.735+0000 7fc605800640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 0x7fc5e006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000f60 tx=0x7fc5f4007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.742+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5f8010040 con 0x7fc6001461f0 2024-04-12T19:51:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.980+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc600142590 con 0x7fc6001461f0 2024-04-12T19:51:51.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.982+0000 7fc5fd600640 1 -- 172.21.15.32:0/1038052091 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7fc5f805aa20 con 0x7fc6001461f0 2024-04-12T19:51:51.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:51.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:38.305869+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.984+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 msgr2=0x7fc5e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.984+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 0x7fc5e006edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000f60 tx=0x7fc5f4007ce0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.984+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 msgr2=0x7fc6000aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.984+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800efc0 tx=0x7fc5f8009f90 comp rx=0 tx=0).stop 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 shutdown_connections 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc5e006c8f0 0x7fc5e006edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6001461f0 0x7fc6000aec30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6000cefc0 0x7fc6000c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 --2- 172.21.15.32:0/1038052091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6000b3e10 0x7fc6000c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 >> 172.21.15.32:0/1038052091 conn(0x7fc6000b27b0 msgr2=0x7fc60006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 shutdown_connections 2024-04-12T19:51:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:51.985+0000 7fc608200640 1 -- 172.21.15.32:0/1038052091 wait complete. 2024-04-12T19:51:52.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:51:34.775305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:51:35.764856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:52.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.231+0000 7efddbe00640 1 Processor -- start 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.231+0000 7efddbe00640 1 -- start start 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efddc15d8f0 0x7efddc0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddbe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc0cecd0 con 0x7efddc0ce850 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc0cefb0 con 0x7efddc15d510 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efddc06dfa0 con 0x7efddc15d8f0 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddb400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efddc0ce850 0x7efddc0b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53940/0 (socket says 172.21.15.32:53940) 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.233+0000 7efddb400640 1 -- 172.21.15.32:0/1305181554 learned_addr learned my addr 172.21.15.32:0/1305181554 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdd3e00640 1 --2- 172.21.15.32:0/1305181554 >> v2:172.21.15.169:3300/0 conn(0x7efddc15d8f0 0x7efddc0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258516213 0 0) 0x7efddc0cecd0 con 0x7efddc0ce850 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc0003520 con 0x7efddc0ce850 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3790658624 0 0) 0x7efdc0003520 con 0x7efddc0ce850 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 >> v2:172.21.15.169:3300/0 conn(0x7efddc15d8f0 msgr2=0x7efddc0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 --2- 172.21.15.32:0/1305181554 >> v2:172.21.15.169:3300/0 conn(0x7efddc15d8f0 0x7efddc0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 >> v1:172.21.15.169:6789/0 conn(0x7efddc15d510 legacy=0x7efddc06ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc06e280 con 0x7efddc0ce850 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7efdd40019f0 con 0x7efddc0ce850 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 >> v1:172.21.15.32:6789/0 conn(0x7efddc0ce850 legacy=0x7efddc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.234+0000 7efdda000640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc00039b0 0x7efdc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdda000640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 0x7efdc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdda000640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc00051c0 0x7efdc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efddc06dfa0 con 0x7efdc0004800 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdc000c060 con 0x7efdc00051c0 2024-04-12T19:51:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdc000c260 con 0x7efdc00039b0 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 0x7efdc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efddaa00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc00039b0 0x7efdc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 -- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc00039b0 msgr2=0x7efdc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc00039b0 0x7efdc0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 -- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc00051c0 msgr2=0x7efdc000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc00051c0 0x7efdc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efdd3e00640 1 -- 172.21.15.32:0/1305181554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc0003520 con 0x7efdc0004800 2024-04-12T19:51:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.235+0000 7efddb400640 1 -- 172.21.15.32:0/1305181554 reap_dead start 2024-04-12T19:51:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.236+0000 7efdd3e00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 0x7efdc0004c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009ac0 tx=0x7efdc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50d5e64ad3197e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4039070 con 0x7efdc0004800 2024-04-12T19:51:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc402c710 con 0x7efdc0004800 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efdda000640 1 -- 172.21.15.32:0/1305181554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4009690 con 0x7efdc0004800 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efddbe00640 1 -- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 msgr2=0x7efdc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efddbe00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 0x7efdc0004c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009ac0 tx=0x7efdc0005830 comp rx=0 tx=0).stop 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efddbe00640 1 -- 172.21.15.32:0/1305181554 shutdown_connections 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efddbe00640 1 --2- 172.21.15.32:0/1305181554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc0004800 0x7efdc0004c80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.237+0000 7efddbe00640 1 -- 172.21.15.32:0/1305181554 >> 172.21.15.32:0/1305181554 conn(0x7efddc0b23a0 msgr2=0x7efddc0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.238+0000 7efddbe00640 1 -- 172.21.15.32:0/1305181554 shutdown_connections 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.238+0000 7efddbe00640 1 -- 172.21.15.32:0/1305181554 wait complete. 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.239+0000 7efddbe00640 1 Processor -- start 2024-04-12T19:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.239+0000 7efddbe00640 1 -- start start 2024-04-12T19:51:52.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc0004800 0x7efddc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 0x7efddc149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0ce490 con 0x7efddc1e7cf0 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0ce310 con 0x7efddc1475c0 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.240+0000 7efddbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efddc0ce610 con 0x7efdc0004800 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58358/0 (socket says 172.21.15.32:58358) 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 -- 172.21.15.32:0/213786786 learned_addr learned my addr 172.21.15.32:0/213786786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efddaa00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc0004800 0x7efddc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efddb400640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 0x7efddc149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 -- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc0004800 msgr2=0x7efddc0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc0004800 0x7efddc0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 -- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 msgr2=0x7efddc149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 0x7efddc149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efdd3e00640 1 -- 172.21.15.32:0/213786786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc1e6f30 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.241+0000 7efddb400640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 0x7efddc149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efdd3e00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdc4000c00 tx=0x7efdc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4039050 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc4045070 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40044e0 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc1e7210 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.242+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1e77d0 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.244+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc1479d0 con 0x7efddc1e7cf0 2024-04-12T19:51:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.244+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdc4003a50 con 0x7efddc1e7cf0 2024-04-12T19:51:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.245+0000 7efdd2a00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 0x7efdb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.245+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7efdc40ad740 con 0x7efddc1e7cf0 2024-04-12T19:51:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.246+0000 7efddaa00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 0x7efdb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.246+0000 7efddaa00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 0x7efdb806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdcc000c00 tx=0x7efdcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.250+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdc404e020 con 0x7efddc1e7cf0 2024-04-12T19:51:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.461+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efddc0755c0 con 0x7efddc1e7cf0 2024-04-12T19:51:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.462+0000 7efdd2a00640 1 -- 172.21.15.32:0/213786786 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7efdc407bb60 con 0x7efddc1e7cf0 2024-04-12T19:51:52.462 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 msgr2=0x7efdb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 0x7efdb806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdcc000c00 tx=0x7efdcc009040 comp rx=0 tx=0).stop 2024-04-12T19:51:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 msgr2=0x7efddc147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdc4000c00 tx=0x7efdc4004730 comp rx=0 tx=0).stop 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 shutdown_connections 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efdb806c8f0 0x7efdb806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efddc1475c0 0x7efddc149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efddc1e7cf0 0x7efddc147080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 --2- 172.21.15.32:0/213786786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc0004800 0x7efddc0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 >> 172.21.15.32:0/213786786 conn(0x7efddc0b23a0 msgr2=0x7efddc0b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 shutdown_connections 2024-04-12T19:51:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.465+0000 7efddbe00640 1 -- 172.21.15.32:0/213786786 wait complete. 2024-04-12T19:51:52.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:52.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.718+0000 7f971e000640 1 Processor -- start 2024-04-12T19:51:52.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.718+0000 7f971e000640 1 -- start start 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9718162b80 0x7f97180b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9718162f60 0x7f971806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971e000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9718072810 con 0x7f97180b4390 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971e000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f97180725f0 con 0x7f9718162f60 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f97180b49c0 con 0x7f9718162b80 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f9717e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97180b4390 0x7f9718071ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53944/0 (socket says 172.21.15.32:53944) 2024-04-12T19:51:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f9717e00640 1 -- 172.21.15.32:0/2622880218 learned_addr learned my addr 172.21.15.32:0/2622880218 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971d600640 1 --2- 172.21.15.32:0/2622880218 >> v2:172.21.15.32:3300/0 conn(0x7f9718162f60 0x7f971806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.720+0000 7f971cc00640 1 --2- 172.21.15.32:0/2622880218 >> v2:172.21.15.169:3300/0 conn(0x7f9718162b80 0x7f97180b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f9717400640 1 -- 172.21.15.32:0/2622880218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335935535 0 0) 0x7f9718072810 con 0x7f97180b4390 2024-04-12T19:51:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f9717400640 1 -- 172.21.15.32:0/2622880218 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9700003520 con 0x7f97180b4390 2024-04-12T19:51:52.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f971d600640 1 -- 172.21.15.32:0/2622880218 >> v2:172.21.15.169:3300/0 conn(0x7f9718162b80 msgr2=0x7f97180b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:52.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f971d600640 1 --2- 172.21.15.32:0/2622880218 >> v2:172.21.15.169:3300/0 conn(0x7f9718162b80 0x7f97180b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f971d600640 1 -- 172.21.15.32:0/2622880218 >> v1:172.21.15.32:6789/0 conn(0x7f97180b4390 legacy=0x7f9718071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.721+0000 7f971d600640 1 -- 172.21.15.32:0/2622880218 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97180b4ca0 con 0x7f9718162f60 2024-04-12T19:51:52.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.722+0000 7f971d600640 1 --2- 172.21.15.32:0/2622880218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9718162f60 0x7f971806f210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97180b61c0 tx=0x7f971002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f239493e550df22a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.722+0000 7f9717400640 1 -- 172.21.15.32:0/2622880218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710039070 con 0x7f9718162f60 2024-04-12T19:51:52.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.722+0000 7f9717400640 1 -- 172.21.15.32:0/2622880218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971002cc70 con 0x7f9718162f60 2024-04-12T19:51:52.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.722+0000 7f9717400640 1 -- 172.21.15.32:0/2622880218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710034660 con 0x7f9718162f60 2024-04-12T19:51:52.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 -- 172.21.15.32:0/2622880218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9718162f60 msgr2=0x7f971806f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 --2- 172.21.15.32:0/2622880218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9718162f60 0x7f971806f210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97180b61c0 tx=0x7f971002bec0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 -- 172.21.15.32:0/2622880218 shutdown_connections 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 --2- 172.21.15.32:0/2622880218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9718162f60 0x7f971806f210 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 --2- 172.21.15.32:0/2622880218 >> v2:172.21.15.169:3300/0 conn(0x7f9718162b80 0x7f97180b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 -- 172.21.15.32:0/2622880218 >> 172.21.15.32:0/2622880218 conn(0x7f971806b930 msgr2=0x7f971806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 -- 172.21.15.32:0/2622880218 shutdown_connections 2024-04-12T19:51:52.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.723+0000 7f971e000640 1 -- 172.21.15.32:0/2622880218 wait complete. 2024-04-12T19:51:52.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.725+0000 7f971e000640 1 Processor -- start 2024-04-12T19:51:52.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.725+0000 7f971e000640 1 -- start start 2024-04-12T19:51:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9718162b80 0x7f97180b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 0x7f97181eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f971806f750 con 0x7f97180b4390 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97180b5260 con 0x7f9718162f60 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f971806f8d0 con 0x7f9718162b80 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58370/0 (socket says 172.21.15.32:58370) 2024-04-12T19:51:52.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.726+0000 7f971cc00640 1 -- 172.21.15.32:0/4105197895 learned_addr learned my addr 172.21.15.32:0/4105197895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971d600640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 0x7f97181eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f9717e00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9718162b80 0x7f97180b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 -- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9718162b80 msgr2=0x7f97180b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9718162b80 0x7f97180b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 -- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 msgr2=0x7f97181eb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 0x7f97181eb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 -- 172.21.15.32:0/4105197895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97180ac6e0 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971d600640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 0x7f97181eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.727+0000 7f971cc00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e9b0 tx=0x7f970c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.728+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c017070 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.728+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c00c9a0 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.728+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c00cd90 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.728+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97180ac9c0 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.728+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97180acf80 con 0x7f97180b4390 2024-04-12T19:51:52.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.729+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97180acb90 con 0x7f97180b4390 2024-04-12T19:51:52.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.730+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f970c01a070 con 0x7f97180b4390 2024-04-12T19:51:52.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.731+0000 7f9714c00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 0x7f96f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.732+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f970c08bf30 con 0x7f97180b4390 2024-04-12T19:51:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.732+0000 7f9717e00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 0x7f96f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:52.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.732+0000 7f9717e00640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 0x7f96f806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f97181fbc90 tx=0x7f9704009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:52.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.737+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f970c05a350 con 0x7f97180b4390 2024-04-12T19:51:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.944+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f971806c8c0 con 0x7f97180b4390 2024-04-12T19:51:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.945+0000 7f9714c00640 1 -- 172.21.15.32:0/4105197895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f970c059c50 con 0x7f97180b4390 2024-04-12T19:51:52.945 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 msgr2=0x7f96f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 0x7f96f806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f97181fbc90 tx=0x7f9704009040 comp rx=0 tx=0).stop 2024-04-12T19:51:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 msgr2=0x7f97180aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e9b0 tx=0x7f970c00ee70 comp rx=0 tx=0).stop 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 shutdown_connections 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f96f806c8f0 0x7f96f806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9718162f60 0x7f97181eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9718162b80 0x7f97180b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 --2- 172.21.15.32:0/4105197895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97180b4390 0x7f97180aefb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 >> 172.21.15.32:0/4105197895 conn(0x7f971806b930 msgr2=0x7f9718161cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 shutdown_connections 2024-04-12T19:51:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:52.948+0000 7f971e000640 1 -- 172.21.15.32:0/4105197895 wait complete. 2024-04-12T19:51:52.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.193+0000 7fe84aa00640 1 Processor -- start 2024-04-12T19:51:53.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.193+0000 7fe84aa00640 1 -- start start 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe84415c3e0 0x7fe8440cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 0x7fe844072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe84415fe50 con 0x7fe8440cfc30 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe8440ced60 con 0x7fe84415c3e0 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe844070800 con 0x7fe8440b0e30 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe848c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8440cfc30 0x7fe8440b0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53954/0 (socket says 172.21.15.32:53954) 2024-04-12T19:51:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe848c00640 1 -- 172.21.15.32:0/2228983330 learned_addr learned my addr 172.21.15.32:0/2228983330 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe849600640 1 --2- 172.21.15.32:0/2228983330 >> v2:172.21.15.169:3300/0 conn(0x7fe84415c3e0 0x7fe8440cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.195+0000 7fe84a000640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 0x7fe844072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261851367 0 0) 0x7fe84415fe50 con 0x7fe8440cfc30 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe82c003520 con 0x7fe8440cfc30 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1908942178 0 0) 0x7fe82c003520 con 0x7fe8440cfc30 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 msgr2=0x7fe844072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 0x7fe844072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 >> v2:172.21.15.169:3300/0 conn(0x7fe84415c3e0 msgr2=0x7fe8440cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 --2- 172.21.15.32:0/2228983330 >> v2:172.21.15.169:3300/0 conn(0x7fe84415c3e0 0x7fe8440cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84415f090 con 0x7fe8440cfc30 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fe830001670 con 0x7fe8440cfc30 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 >> v1:172.21.15.32:6789/0 conn(0x7fe8440cfc30 legacy=0x7fe8440b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82c0039b0 0x7fe82c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 0x7fe82c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c004d40 0x7fe82c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c00b9c0 con 0x7fe82c0039b0 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c00bb40 con 0x7fe82c004d40 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe82c00bcc0 con 0x7fe8440b0e30 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.196+0000 7fe849600640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82c0039b0 0x7fe82c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe849600640 1 -- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 msgr2=0x7fe82c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe849600640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8440b0e30 0x7fe82c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe849600640 1 -- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c004d40 msgr2=0x7fe82c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe84a000640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c004d40 0x7fe82c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe849600640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c004d40 0x7fe82c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.197+0000 7fe849600640 1 -- 172.21.15.32:0/2228983330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c003520 con 0x7fe82c0039b0 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.198+0000 7fe849600640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82c0039b0 0x7fe82c003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe838009ac0 tx=0x7fe82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500d2089db735e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.198+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838039070 con 0x7fe82c0039b0 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.198+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83802c710 con 0x7fe82c0039b0 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.198+0000 7fe843e00640 1 -- 172.21.15.32:0/2228983330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838009690 con 0x7fe82c0039b0 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 -- 172.21.15.32:0/2228983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82c0039b0 msgr2=0x7fe82c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 --2- 172.21.15.32:0/2228983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82c0039b0 0x7fe82c003dd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe838009ac0 tx=0x7fe82c005830 comp rx=0 tx=0).stop 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84a000640 1 -- 172.21.15.32:0/2228983330 reap_dead start 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 -- 172.21.15.32:0/2228983330 shutdown_connections 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 -- 172.21.15.32:0/2228983330 >> 172.21.15.32:0/2228983330 conn(0x7fe84406f0c0 msgr2=0x7fe84406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 -- 172.21.15.32:0/2228983330 shutdown_connections 2024-04-12T19:51:53.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.199+0000 7fe84aa00640 1 -- 172.21.15.32:0/2228983330 wait complete. 2024-04-12T19:51:53.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.201+0000 7fe84aa00640 1 Processor -- start 2024-04-12T19:51:53.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.201+0000 7fe84aa00640 1 -- start start 2024-04-12T19:51:53.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c0039b0 0x7fe84414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe844159300 0x7fe844157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe84415f650 con 0x7fe844157880 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8440d0070 con 0x7fe82c0039b0 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe8440d02b0 con 0x7fe844159300 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58380/0 (socket says 172.21.15.32:58380) 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.202+0000 7fe84a000640 1 -- 172.21.15.32:0/3599085649 learned_addr learned my addr 172.21.15.32:0/3599085649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe849600640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c0039b0 0x7fe84414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 -- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe844159300 msgr2=0x7fe844157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe844159300 0x7fe844157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 -- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c0039b0 msgr2=0x7fe84414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c0039b0 0x7fe84414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 -- 172.21.15.32:0/3599085649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8440cad40 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe84a000640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8440d07b0 tx=0x7fe83c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.203+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c01c070 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.204+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83c00fe50 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.204+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8440cb020 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.204+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c017640 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.204+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8441f7c40 con 0x7fe844157880 2024-04-12T19:51:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.205+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8441f7e40 con 0x7fe844157880 2024-04-12T19:51:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.206+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe83c01f070 con 0x7fe844157880 2024-04-12T19:51:53.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.207+0000 7fe841600640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.207+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe83c08bb80 con 0x7fe844157880 2024-04-12T19:51:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.208+0000 7fe849600640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.208+0000 7fe849600640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 0x7fe82406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe838036000 tx=0x7fe838030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.212+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8441f7e40 con 0x7fe844157880 2024-04-12T19:51:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.425+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8440cefe0 con 0x7fe844157880 2024-04-12T19:51:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.426+0000 7fe841600640 1 -- 172.21.15.32:0/3599085649 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fe83c012070 con 0x7fe844157880 2024-04-12T19:51:53.426 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.428+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 msgr2=0x7fe82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.428+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 0x7fe82406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe838036000 tx=0x7fe838030000 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.428+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 msgr2=0x7fe8441f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.428+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8440d07b0 tx=0x7fe83c00f4a0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 shutdown_connections 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fe82406c8f0 0x7fe82406edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe844157880 0x7fe8441f8160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe844159300 0x7fe844157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 --2- 172.21.15.32:0/3599085649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe82c0039b0 0x7fe84414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 >> 172.21.15.32:0/3599085649 conn(0x7fe84406f0c0 msgr2=0x7fe84415b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 shutdown_connections 2024-04-12T19:51:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.429+0000 7fe84aa00640 1 -- 172.21.15.32:0/3599085649 wait complete. 2024-04-12T19:51:53.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T19:51:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.668+0000 7f35e8a00640 1 Processor -- start 2024-04-12T19:51:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.668+0000 7f35e8a00640 1 -- start start 2024-04-12T19:51:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e8a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35e413f6a0 0x7f35e40ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40725b0 0x7f35e406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e4072a20 con 0x7f35e4072100 2024-04-12T19:51:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e8a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f35e40a95b0 con 0x7f35e413f6a0 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35e415d200 con 0x7f35e40725b0 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35e413f6a0 0x7f35e40ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35e413f6a0 0x7f35e40ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58394/0 (socket says 172.21.15.32:58394) 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.670+0000 7f35e1e00640 1 -- 172.21.15.32:0/818423381 learned_addr learned my addr 172.21.15.32:0/818423381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.671+0000 7f35e2800640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40725b0 0x7f35e406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.671+0000 7f35e1e00640 1 -- 172.21.15.32:0/818423381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40725b0 msgr2=0x7f35e406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.671+0000 7f35e1e00640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40725b0 0x7f35e406e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.671+0000 7f35e1e00640 1 -- 172.21.15.32:0/818423381 >> v1:172.21.15.169:6789/0 conn(0x7f35e4072100 legacy=0x7f35e406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.671+0000 7f35e1e00640 1 -- 172.21.15.32:0/818423381 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e415d4e0 con 0x7f35e413f6a0 2024-04-12T19:51:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.672+0000 7f35e1e00640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e413f6a0 0x7f35e40ceda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707b8dafbd4cfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.672+0000 7f35e0a00640 1 -- 172.21.15.32:0/818423381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4034070 con 0x7f35e413f6a0 2024-04-12T19:51:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.672+0000 7f35e0a00640 1 -- 172.21.15.32:0/818423381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4030840 con 0x7f35e413f6a0 2024-04-12T19:51:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.672+0000 7f35e0a00640 1 -- 172.21.15.32:0/818423381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4030c10 con 0x7f35e413f6a0 2024-04-12T19:51:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 -- 172.21.15.32:0/818423381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e413f6a0 msgr2=0x7f35e40ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e413f6a0 0x7f35e40ceda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d402b980 comp rx=0 tx=0).stop 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 -- 172.21.15.32:0/818423381 shutdown_connections 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e40725b0 0x7f35e406e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 --2- 172.21.15.32:0/818423381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e413f6a0 0x7f35e40ceda0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 -- 172.21.15.32:0/818423381 >> 172.21.15.32:0/818423381 conn(0x7f35e40b2460 msgr2=0x7f35e40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 -- 172.21.15.32:0/818423381 shutdown_connections 2024-04-12T19:51:53.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.673+0000 7f35e8a00640 1 -- 172.21.15.32:0/818423381 wait complete. 2024-04-12T19:51:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.675+0000 7f35e8a00640 1 Processor -- start 2024-04-12T19:51:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.675+0000 7f35e8a00640 1 -- start start 2024-04-12T19:51:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40725b0 0x7f35e40adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e413f6a0 0x7f35e40ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35e413fef0 con 0x7f35e4072100 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4070400 con 0x7f35e40725b0 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.676+0000 7f35e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35e415dd80 con 0x7f35e413f6a0 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58408/0 (socket says 172.21.15.32:58408) 2024-04-12T19:51:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 -- 172.21.15.32:0/3871747970 learned_addr learned my addr 172.21.15.32:0/3871747970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e2800640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e413f6a0 0x7f35e40ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 -- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e413f6a0 msgr2=0x7f35e40ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1400640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40725b0 0x7f35e40adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e413f6a0 0x7f35e40ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 -- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40725b0 msgr2=0x7f35e40adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40725b0 0x7f35e40adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 -- 172.21.15.32:0/3871747970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e40b1cb0 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.677+0000 7f35e1e00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.678+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4048070 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.678+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4030640 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.678+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4034040 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.678+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e40b1f90 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.678+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e4137c50 con 0x7f35e4072100 2024-04-12T19:51:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.679+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e4068c80 con 0x7f35e4072100 2024-04-12T19:51:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.680+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35d4037070 con 0x7f35e4072100 2024-04-12T19:51:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.681+0000 7f35da000640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.681+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f35d40ad950 con 0x7f35e4072100 2024-04-12T19:51:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.681+0000 7f35e1400640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.682+0000 7f35e1400640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 0x7f35c406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35e40cd4c0 tx=0x7f35d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.686+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35d407bf30 con 0x7f35e4072100 2024-04-12T19:51:53.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.904+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35e4157270 con 0x7f35e4072100 2024-04-12T19:51:53.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.905+0000 7f35da000640 1 -- 172.21.15.32:0/3871747970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f35d407b830 con 0x7f35e4072100 2024-04-12T19:51:53.905 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-12T19:51:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 msgr2=0x7f35c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 0x7f35c406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35e40cd4c0 tx=0x7f35d0008040 comp rx=0 tx=0).stop 2024-04-12T19:51:53.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 msgr2=0x7f35e40c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d4004730 comp rx=0 tx=0).stop 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 shutdown_connections 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35c406c8f0 0x7f35c406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35e413f6a0 0x7f35e40ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35e40725b0 0x7f35e40adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 --2- 172.21.15.32:0/3871747970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e4072100 0x7f35e40c56d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 >> 172.21.15.32:0/3871747970 conn(0x7f35e40b2460 msgr2=0x7f35e40b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 shutdown_connections 2024-04-12T19:51:53.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:53.908+0000 7f35e8a00640 1 -- 172.21.15.32:0/3871747970 wait complete. 2024-04-12T19:51:53.929 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-12T19:51:53.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:53.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-12T19:51:54.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:51:54.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:51:54.084 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-12T19:51:54.136 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-04-12T19:51:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.218+0000 7f2f25436f40 1 Processor -- start 2024-04-12T19:51:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 -- start start 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080030f0 0x55677bdbce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be5de30 0x55677bdda3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55677bd35c90 con 0x55677bdbe110 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55677bd39cd0 con 0x7f2f080030f0 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.219+0000 7f2f25436f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55677bd8d350 con 0x55677be5de30 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41832/0 (socket says 172.21.15.32:41832) 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 -- 172.21.15.32:0/2142716039 learned_addr learned my addr 172.21.15.32:0/2142716039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 -- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be5de30 msgr2=0x55677bdda3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be5de30 0x55677bdda3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 -- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080030f0 msgr2=0x55677bdbce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080030f0 0x55677bdbce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 -- 172.21.15.32:0/2142716039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677bd8f020 con 0x55677bdbe110 2024-04-12T19:51:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.220+0000 7f2f21200640 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009840 tx=0x7f2f10002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dfe048ac7d8fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f1b400640 1 -- 172.21.15.32:0/2142716039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10061070 con 0x55677bdbe110 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f1b400640 1 -- 172.21.15.32:0/2142716039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f100529f0 con 0x55677bdbe110 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f1b400640 1 -- 172.21.15.32:0/2142716039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10057040 con 0x55677bdbe110 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 -- 172.21.15.32:0/2142716039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 msgr2=0x55677bdd9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009840 tx=0x7f2f10002fb0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 -- 172.21.15.32:0/2142716039 shutdown_connections 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be5de30 0x55677bdda3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55677bdbe110 0x55677bdd9ea0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 --2- 172.21.15.32:0/2142716039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f080030f0 0x55677bdbce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.221+0000 7f2f25436f40 1 -- 172.21.15.32:0/2142716039 >> 172.21.15.32:0/2142716039 conn(0x55677bdbdab0 msgr2=0x55677bd8e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.222+0000 7f2f25436f40 1 -- 172.21.15.32:0/2142716039 shutdown_connections 2024-04-12T19:51:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.222+0000 7f2f25436f40 1 -- 172.21.15.32:0/2142716039 wait complete. 2024-04-12T19:51:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.222+0000 7f2f25436f40 1 Processor -- start 2024-04-12T19:51:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 -- start start 2024-04-12T19:51:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55677be5de30 0x55677be5cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 0x55677be49610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be4a260 0x55677bef04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55677be5ef40 con 0x7f2f080030f0 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55677bdbb8a0 con 0x55677be5de30 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.223+0000 7f2f25436f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55677bdbc0c0 con 0x55677be4a260 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 0x55677be49610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 0x55677be49610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41834/0 (socket says 172.21.15.32:41834) 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 -- 172.21.15.32:0/1619727550 learned_addr learned my addr 172.21.15.32:0/1619727550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21c00640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55677be5de30 0x55677be5cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 -- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be4a260 msgr2=0x55677bef04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55677be4a260 0x55677bef04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 -- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55677be5de30 msgr2=0x55677be5cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55677be5de30 0x55677be5cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 -- 172.21.15.32:0/1619727550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677beef150 con 0x7f2f080030f0 2024-04-12T19:51:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21c00640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55677be5de30 0x55677be5cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.224+0000 7f2f21200640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 0x55677be49610 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f10055040 tx=0x7f2f10005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.225+0000 7f2f1a000640 1 -- 172.21.15.32:0/1619727550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10057070 con 0x7f2f080030f0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.225+0000 7f2f1a000640 1 -- 172.21.15.32:0/1619727550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f100527f0 con 0x7f2f080030f0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.225+0000 7f2f1a000640 1 -- 172.21.15.32:0/1619727550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10061040 con 0x7f2f080030f0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.225+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55677beef430 con 0x7f2f080030f0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.225+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677be5d000 con 0x7f2f080030f0 2024-04-12T19:51:54.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.227+0000 7f2f1a000640 1 -- 172.21.15.32:0/1619727550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f1006c020 con 0x7f2f080030f0 2024-04-12T19:51:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.228+0000 7f2f1a000640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f0406b2d0 0x7f2f0406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.228+0000 7f2f1a000640 1 -- 172.21.15.32:0/1619727550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2f100cc990 con 0x7f2f080030f0 2024-04-12T19:51:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.228+0000 7f2f25436f40 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55677bef2530 0x55677bef29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.228+0000 7f2f21c00640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f0406b2d0 0x7f2f0406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.228+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55677bef2ee0 con 0x55677bef2530 2024-04-12T19:51:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.229+0000 7f2f22600640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55677bef2530 0x55677bef29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.229+0000 7f2f21c00640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f0406b2d0 0x7f2f0406d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002e70 tx=0x7f2f14002450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.230+0000 7f2f22600640 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55677bef2530 0x55677bef29a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f22600640 1 -- 172.21.15.32:0/1619727550 <== osd.7 v2:172.21.15.169:6822/883923800 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x7f2f1c00e1d0 con 0x55677bef2530 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:error getting omap key set cephfs_metadata/10000000000.00000000: (2) No such file or directory 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55677bef2530 msgr2=0x55677bef29a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55677bef2530 0x55677bef29a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f0406b2d0 msgr2=0x7f2f0406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2f0406b2d0 0x7f2f0406d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002e70 tx=0x7f2f14002450 comp rx=0 tx=0).stop 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 msgr2=0x55677be49610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 --2- 172.21.15.32:0/1619727550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f080030f0 0x55677be49610 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f10055040 tx=0x7f2f10005a00 comp rx=0 tx=0).stop 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f22600640 1 -- 172.21.15.32:0/1619727550 reap_dead start 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 shutdown_connections 2024-04-12T19:51:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 >> 172.21.15.32:0/1619727550 conn(0x55677bdbdab0 msgr2=0x55677bdda840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 shutdown_connections 2024-04-12T19:51:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.231+0000 7f2f25436f40 1 -- 172.21.15.32:0/1619727550 wait complete. 2024-04-12T19:51:54.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:51:54.236 ERROR:tasks.cephfs.filesystem:Command failed on smithi032 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000' 2024-04-12T19:51:54.237 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-12T19:51:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.322+0000 7f4cc2246f40 1 Processor -- start 2024-04-12T19:51:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.322+0000 7f4cc2246f40 1 -- start start 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fa4c8f0 0x55707fad23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fad67b0 0x55707fad44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55707fa180b0 con 0x55707fa178e0 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55707f9c0c90 con 0x55707fa4c8f0 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55707fa18350 con 0x55707fad67b0 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41842/0 (socket says 172.21.15.32:41842) 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 -- 172.21.15.32:0/3375012470 learned_addr learned my addr 172.21.15.32:0/3375012470 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 -- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fad67b0 msgr2=0x55707fad44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fad67b0 0x55707fad44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 -- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fa4c8f0 msgr2=0x55707fad23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fa4c8f0 0x55707fad23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.323+0000 7f4cbea00640 1 -- 172.21.15.32:0/3375012470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707fa46a40 con 0x55707fa178e0 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cbea00640 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001680 tx=0x7f4cb0001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d667de29472ad server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cbcc00640 1 -- 172.21.15.32:0/3375012470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003650 con 0x55707fa178e0 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cbcc00640 1 -- 172.21.15.32:0/3375012470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb00037f0 con 0x55707fa178e0 2024-04-12T19:51:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cbcc00640 1 -- 172.21.15.32:0/3375012470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003be0 con 0x55707fa178e0 2024-04-12T19:51:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3375012470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 msgr2=0x55707fad58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001680 tx=0x7f4cb0001b40 comp rx=0 tx=0).stop 2024-04-12T19:51:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3375012470 shutdown_connections 2024-04-12T19:51:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fad67b0 0x55707fad44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fa4c8f0 0x55707fad23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3375012470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa178e0 0x55707fad58f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.324+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3375012470 >> 172.21.15.32:0/3375012470 conn(0x55707fa48ab0 msgr2=0x55707fad3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.325+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3375012470 shutdown_connections 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.325+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3375012470 wait complete. 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.325+0000 7f4cc2246f40 1 Processor -- start 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 -- start start 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fa178e0 0x55707fae82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 0x55707fa474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fad67b0 0x55707fad1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55707fae8fc0 con 0x55707fa4c8f0 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55707fad5460 con 0x55707fad67b0 2024-04-12T19:51:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.326+0000 7f4cc2246f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55707fa46ce0 con 0x55707fa178e0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 0x55707fa474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 0x55707fa474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41852/0 (socket says 172.21.15.32:41852) 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 -- 172.21.15.32:0/3258522239 learned_addr learned my addr 172.21.15.32:0/3258522239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 -- 172.21.15.32:0/3258522239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fa178e0 msgr2=0x55707fae82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55707fa178e0 0x55707fae82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 -- 172.21.15.32:0/3258522239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fad67b0 msgr2=0x55707fad1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbf400640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fad67b0 0x55707fad1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55707fad67b0 0x55707fad1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 -- 172.21.15.32:0/3258522239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707faf2350 con 0x55707fa4c8f0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.327+0000 7f4cbe000640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 0x55707fa474b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800aea0 tx=0x7f4ca800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.328+0000 7f4cb7400640 1 -- 172.21.15.32:0/3258522239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca801a070 con 0x55707fa4c8f0 2024-04-12T19:51:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.328+0000 7f4cb7400640 1 -- 172.21.15.32:0/3258522239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca8008b60 con 0x55707fa4c8f0 2024-04-12T19:51:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.328+0000 7f4cb7400640 1 -- 172.21.15.32:0/3258522239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8015420 con 0x55707fa4c8f0 2024-04-12T19:51:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.328+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55707faf2630 con 0x55707fa4c8f0 2024-04-12T19:51:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.328+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707faf2bf0 con 0x55707fa4c8f0 2024-04-12T19:51:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.330+0000 7f4cb7400640 1 -- 172.21.15.32:0/3258522239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ca801d070 con 0x55707fa4c8f0 2024-04-12T19:51:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cb7400640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4c9c06b2d0 0x7f4c9c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cb7400640 1 -- 172.21.15.32:0/3258522239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4ca8089c90 con 0x55707fa4c8f0 2024-04-12T19:51:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cbea00640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4c9c06b2d0 0x7f4c9c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x55707fad1850 0x55707faf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55707faf11c0 con 0x55707fad1850 2024-04-12T19:51:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.331+0000 7f4cbf400640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x55707fad1850 0x55707faf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.332+0000 7f4cbea00640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4c9c06b2d0 0x7f4c9c06d790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001680 tx=0x7f4cb000e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.332+0000 7f4cbf400640 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x55707fad1850 0x55707faf0c80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cbf400640 1 -- 172.21.15.32:0/3258522239 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x7f4cb8050990 con 0x55707fad1850 2024-04-12T19:51:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:error getting xattr cephfs_data/10000000001.00000000/parent: (2) No such file or directory 2024-04-12T19:51:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x55707fad1850 msgr2=0x55707faf0c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x55707fad1850 0x55707faf0c80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4c9c06b2d0 msgr2=0x7f4c9c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4c9c06b2d0 0x7f4c9c06d790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001680 tx=0x7f4cb000e000 comp rx=0 tx=0).stop 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 msgr2=0x55707fa474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 --2- 172.21.15.32:0/3258522239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55707fa4c8f0 0x55707fa474b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800aea0 tx=0x7f4ca800af90 comp rx=0 tx=0).stop 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cbf400640 1 -- 172.21.15.32:0/3258522239 reap_dead start 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 shutdown_connections 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 >> 172.21.15.32:0/3258522239 conn(0x55707fa48ab0 msgr2=0x55707fad23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 shutdown_connections 2024-04-12T19:51:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.333+0000 7f4cc2246f40 1 -- 172.21.15.32:0/3258522239 wait complete. 2024-04-12T19:51:54.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:51:54.338 ERROR:tasks.cephfs.filesystem:Command failed on smithi032 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent' 2024-04-12T19:51:54.339 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-04-12T19:51:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.429+0000 7f4164a0cf40 1 Processor -- start 2024-04-12T19:51:54.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 -- start start 2024-04-12T19:51:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264b1c720 0x564264ba4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564264ae80b0 con 0x564264ae78e0 2024-04-12T19:51:54.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564264b1cb30 con 0x564264ba67b0 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.430+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564264ae8350 con 0x564264b1c720 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41858/0 (socket says 172.21.15.32:41858) 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 -- 172.21.15.32:0/639515800 learned_addr learned my addr 172.21.15.32:0/639515800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 -- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264b1c720 msgr2=0x564264ba4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264b1c720 0x564264ba4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 -- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 msgr2=0x564264ba5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 -- 172.21.15.32:0/639515800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564264a14b00 con 0x564264ae78e0 2024-04-12T19:51:54.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.431+0000 7f4160e00640 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4150046fe0 tx=0x7f415004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0be2fb969687a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f415b400640 1 -- 172.21.15.32:0/639515800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415005a040 con 0x564264ae78e0 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f415b400640 1 -- 172.21.15.32:0/639515800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4150065070 con 0x564264ae78e0 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f415b400640 1 -- 172.21.15.32:0/639515800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415004dd50 con 0x564264ae78e0 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 -- 172.21.15.32:0/639515800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 msgr2=0x564264ba2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4150046fe0 tx=0x7f415004f000 comp rx=0 tx=0).stop 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 -- 172.21.15.32:0/639515800 shutdown_connections 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264b1c720 0x564264ba4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/639515800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264ae78e0 0x564264ba2530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.432+0000 7f4164a0cf40 1 -- 172.21.15.32:0/639515800 >> 172.21.15.32:0/639515800 conn(0x564264b18ab0 msgr2=0x564264ba2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.433+0000 7f4164a0cf40 1 -- 172.21.15.32:0/639515800 shutdown_connections 2024-04-12T19:51:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.433+0000 7f4164a0cf40 1 -- 172.21.15.32:0/639515800 wait complete. 2024-04-12T19:51:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.433+0000 7f4164a0cf40 1 Processor -- start 2024-04-12T19:51:54.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 -- start start 2024-04-12T19:51:54.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264ae78e0 0x564264ba1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 0x564264c569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564264ba2a70 con 0x564264b1c720 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564264ba4be0 con 0x564264ba67b0 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.434+0000 7f4164a0cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564264b334c0 con 0x564264ae78e0 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 0x564264c569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 0x564264c569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41860/0 (socket says 172.21.15.32:41860) 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 -- 172.21.15.32:0/1136748823 learned_addr learned my addr 172.21.15.32:0/1136748823 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4160e00640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264ae78e0 0x564264ba1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4161800640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 -- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264ae78e0 msgr2=0x564264ba1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264ae78e0 0x564264ba1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 -- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 msgr2=0x564264ba0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564264ba67b0 0x564264ba0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4158400640 1 -- 172.21.15.32:0/1136748823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564264b172c0 con 0x564264b1c720 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.435+0000 7f4160e00640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564264ae78e0 0x564264ba1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:54.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f4158400640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 0x564264c569b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4148008490 tx=0x7f4148008950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f415a000640 1 -- 172.21.15.32:0/1136748823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148016070 con 0x564264b1c720 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f415a000640 1 -- 172.21.15.32:0/1136748823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41480067c0 con 0x564264b1c720 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f415a000640 1 -- 172.21.15.32:0/1136748823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148006bb0 con 0x564264b1c720 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564264c57270 con 0x564264b1c720 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.436+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564264c57830 con 0x564264b1c720 2024-04-12T19:51:54.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.438+0000 7f415a000640 1 -- 172.21.15.32:0/1136748823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4148019070 con 0x564264b1c720 2024-04-12T19:51:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.439+0000 7f415a000640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f413c06b2d0 0x7f413c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.439+0000 7f415a000640 1 -- 172.21.15.32:0/1136748823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f414808a210 con 0x564264b1c720 2024-04-12T19:51:54.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.439+0000 7f4160e00640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f413c06b2d0 0x7f413c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.440+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x564264b17a80 0x564264b17e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.440+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- osd_op(unknown.0.0:1 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x564264b19fc0 con 0x564264b17a80 2024-04-12T19:51:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.440+0000 7f4161800640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x564264b17a80 0x564264b17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.440+0000 7f4160e00640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f413c06b2d0 0x7f413c06d790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4150062040 tx=0x7f4150052000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4161800640 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x564264b17a80 0x564264b17e60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4161800640 1 -- 172.21.15.32:0/1136748823 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 154+0+5 (crc 0 0 0) 0x7f415c00c580 con 0x564264b17a80 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x564264b17a80 msgr2=0x564264b17e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x564264b17a80 0x564264b17e60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f413c06b2d0 msgr2=0x7f413c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f413c06b2d0 0x7f413c06d790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4150062040 tx=0x7f4150052000 comp rx=0 tx=0).stop 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 msgr2=0x564264c569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 --2- 172.21.15.32:0/1136748823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564264b1c720 0x564264c569b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4148008490 tx=0x7f4148008950 comp rx=0 tx=0).stop 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4161800640 1 -- 172.21.15.32:0/1136748823 reap_dead start 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 shutdown_connections 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.441+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 >> 172.21.15.32:0/1136748823 conn(0x564264b18ab0 msgr2=0x564264ba4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.442+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 shutdown_connections 2024-04-12T19:51:54.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.442+0000 7f4164a0cf40 1 -- 172.21.15.32:0/1136748823 wait complete. 2024-04-12T19:51:54.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:54.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.658+0000 7f6b42800640 1 Processor -- start 2024-04-12T19:51:54.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.658+0000 7f6b42800640 1 -- start start 2024-04-12T19:51:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b42800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b3c0cee00 0x7f6b3c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b42800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c15dbb0 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b42800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c15d8d0 con 0x7f6b3c071f50 2024-04-12T19:51:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b42800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3c06f4f0 con 0x7f6b3c0cee00 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b41e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b3c15d2f0 0x7f6b3c15dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46108/0 (socket says 172.21.15.32:46108) 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.660+0000 7f6b41e00640 1 -- 172.21.15.32:0/1667824394 learned_addr learned my addr 172.21.15.32:0/1667824394 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b41400640 1 --2- 172.21.15.32:0/1667824394 >> v2:172.21.15.169:3300/0 conn(0x7f6b3c0cee00 0x7f6b3c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941619502 0 0) 0x7f6b3c15dbb0 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b24003520 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4174517109 0 0) 0x7f6b24003520 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 >> v2:172.21.15.169:3300/0 conn(0x7f6b3c0cee00 msgr2=0x7f6b3c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 --2- 172.21.15.32:0/1667824394 >> v2:172.21.15.169:3300/0 conn(0x7f6b3c0cee00 0x7f6b3c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 >> v1:172.21.15.169:6789/0 conn(0x7f6b3c071f50 legacy=0x7f6b3c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c06dde0 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6b340019f0 con 0x7f6b3c15d2f0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.661+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 >> v1:172.21.15.32:6789/0 conn(0x7f6b3c15d2f0 legacy=0x7f6b3c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24003a00 0x7f6b24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400c090 con 0x7f6b240051c0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2400c240 con 0x7f6b24003a00 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2400c440 con 0x7f6b24004800 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41e00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41400640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24003a00 0x7f6b24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41e00640 1 -- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b24004800 msgr2=0x7f6b24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b40a00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41e00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41e00640 1 -- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24003a00 msgr2=0x7f6b24003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.662+0000 7f6b41e00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24003a00 0x7f6b24003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.663+0000 7f6b41e00640 1 -- 172.21.15.32:0/1667824394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b24003520 con 0x7f6b240051c0 2024-04-12T19:51:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.663+0000 7f6b41400640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b24003a00 0x7f6b24003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:51:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.663+0000 7f6b40a00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b24004800 0x7f6b24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.663+0000 7f6b41e00640 1 -- 172.21.15.32:0/1667824394 reap_dead start 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b41e00640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400a790 tx=0x7f6b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effae9bf44285827 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400a3b0 con 0x7f6b240051c0 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b34034670 con 0x7f6b240051c0 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b3be00640 1 -- 172.21.15.32:0/1667824394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34034a60 con 0x7f6b240051c0 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b42800640 1 -- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 msgr2=0x7f6b2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.664+0000 7f6b42800640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400a790 tx=0x7f6b24005830 comp rx=0 tx=0).stop 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.665+0000 7f6b42800640 1 -- 172.21.15.32:0/1667824394 shutdown_connections 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.665+0000 7f6b42800640 1 --2- 172.21.15.32:0/1667824394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b240051c0 0x7f6b2400b9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.665+0000 7f6b42800640 1 -- 172.21.15.32:0/1667824394 >> 172.21.15.32:0/1667824394 conn(0x7f6b3c06b930 msgr2=0x7f6b3c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.665+0000 7f6b42800640 1 -- 172.21.15.32:0/1667824394 shutdown_connections 2024-04-12T19:51:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.665+0000 7f6b42800640 1 -- 172.21.15.32:0/1667824394 wait complete. 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.667+0000 7f6b42800640 1 Processor -- start 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.667+0000 7f6b42800640 1 -- start start 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b240051c0 0x7f6b3c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 0x7f6b3c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c06e8c0 con 0x7f6b3c1f5170 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c06e3a0 con 0x7f6b3c14fe60 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b42800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c06ea40 con 0x7f6b240051c0 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b40a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b40a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41864/0 (socket says 172.21.15.32:41864) 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.668+0000 7f6b40a00640 1 -- 172.21.15.32:0/175869037 learned_addr learned my addr 172.21.15.32:0/175869037 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b41400640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b240051c0 0x7f6b3c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b41e00640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 0x7f6b3c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 -- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b240051c0 msgr2=0x7f6b3c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b240051c0 0x7f6b3c0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 -- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 msgr2=0x7f6b3c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 0x7f6b3c15fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 -- 172.21.15.32:0/175869037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c1f5650 con 0x7f6b3c1f5170 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b41e00640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 0x7f6b3c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b40a00640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800d7a0 tx=0x7f6b2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28005270 con 0x7f6b3c1f5170 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b28017070 con 0x7f6b3c1f5170 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800b6b0 con 0x7f6b3c1f5170 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1f5930 con 0x7f6b3c1f5170 2024-04-12T19:51:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.669+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1f5ef0 con 0x7f6b3c1f5170 2024-04-12T19:51:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.671+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c0755c0 con 0x7f6b3c1f5170 2024-04-12T19:51:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.672+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2800b850 con 0x7f6b3c1f5170 2024-04-12T19:51:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.672+0000 7f6b39600640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 0x7f6b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.673+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2808ba10 con 0x7f6b3c1f5170 2024-04-12T19:51:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.673+0000 7f6b41400640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 0x7f6b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.673+0000 7f6b41400640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 0x7f6b1c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009990 tx=0x7f6b30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.677+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b28059e40 con 0x7f6b3c1f5170 2024-04-12T19:51:54.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.924+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b3c159f50 con 0x7f6b3c1f5170 2024-04-12T19:51:54.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.925+0000 7f6b39600640 1 -- 172.21.15.32:0/175869037 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f6b28059740 con 0x7f6b3c1f5170 2024-04-12T19:51:54.926 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:54.926 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:51:54.926 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:51:54.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 msgr2=0x7f6b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 0x7f6b1c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009990 tx=0x7f6b30008040 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 msgr2=0x7f6b3c0c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800d7a0 tx=0x7f6b2800dc60 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 shutdown_connections 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b1c06c8f0 0x7f6b1c06edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b3c14fe60 0x7f6b3c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c1f5170 0x7f6b3c0c85b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 --2- 172.21.15.32:0/175869037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b240051c0 0x7f6b3c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.928+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 >> 172.21.15.32:0/175869037 conn(0x7f6b3c06b930 msgr2=0x7f6b3c0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.929+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 shutdown_connections 2024-04-12T19:51:54.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:54.929+0000 7f6b42800640 1 -- 172.21.15.32:0/175869037 wait complete. 2024-04-12T19:51:54.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-12T19:51:55.210 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-12T19:51:55.210 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-12T19:51:55.211 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-04-12T19:51:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.319+0000 7f4ef7492f40 1 Processor -- start 2024-04-12T19:51:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.319+0000 7f4ef7492f40 1 -- start start 2024-04-12T19:51:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d87d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d7dd720 0x55e20d8662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e20d7a9350 con 0x55e20d8677b0 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e20d7a90b0 con 0x55e20d7a88e0 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.320+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e20d7a95f0 con 0x55e20d7dd720 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41896/0 (socket says 172.21.15.32:41896) 2024-04-12T19:51:55.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 -- 172.21.15.32:0/982546589 learned_addr learned my addr 172.21.15.32:0/982546589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4400640 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d87d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4eeba00640 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d7dd720 0x55e20d8662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 -- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d7dd720 msgr2=0x55e20d8662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d7dd720 0x55e20d8662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 -- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 msgr2=0x55e20d87d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d87d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 -- 172.21.15.32:0/982546589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20d6d5b00 con 0x55e20d8677b0 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.321+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4eec01e190 tx=0x7f4eec04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb76eeb60c9f740 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef3000640 1 -- 172.21.15.32:0/982546589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec05a040 con 0x55e20d8677b0 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef3000640 1 -- 172.21.15.32:0/982546589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec064070 con 0x55e20d8677b0 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef3000640 1 -- 172.21.15.32:0/982546589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec04dbe0 con 0x55e20d8677b0 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 -- 172.21.15.32:0/982546589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 msgr2=0x55e20d866830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4eec01e190 tx=0x7f4eec04f000 comp rx=0 tx=0).stop 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 -- 172.21.15.32:0/982546589 shutdown_connections 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d8677b0 0x55e20d866830 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d7dd720 0x55e20d8662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/982546589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d87d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.322+0000 7f4ef7492f40 1 -- 172.21.15.32:0/982546589 >> 172.21.15.32:0/982546589 conn(0x55e20d7d9ab0 msgr2=0x55e20d863c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.323+0000 7f4ef7492f40 1 -- 172.21.15.32:0/982546589 shutdown_connections 2024-04-12T19:51:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.323+0000 7f4ef7492f40 1 -- 172.21.15.32:0/982546589 wait complete. 2024-04-12T19:51:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.323+0000 7f4ef7492f40 1 Processor -- start 2024-04-12T19:51:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 -- start start 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d8839d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 0x55e20d87cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d8677b0 0x55e20d87ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e20d865b10 con 0x55e20d7dd720 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e20d7d8f40 con 0x55e20d7a88e0 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.324+0000 7f4ef7492f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e20d7f7710 con 0x55e20d8677b0 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 0x55e20d87cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 0x55e20d87cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41902/0 (socket says 172.21.15.32:41902) 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 -- 172.21.15.32:0/423850081 learned_addr learned my addr 172.21.15.32:0/423850081 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4ef4400640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d8839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d8677b0 0x55e20d87ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 -- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d8677b0 msgr2=0x55e20d87ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e20d8677b0 0x55e20d87ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 -- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 msgr2=0x55e20d8839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e20d7a88e0 0x55e20d8839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.325+0000 7f4eeba00640 1 -- 172.21.15.32:0/423850081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20d87d980 con 0x55e20d7dd720 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4eeba00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 0x55e20d87cf50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007dd0 tx=0x7f4edc006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4ef1c00640 1 -- 172.21.15.32:0/423850081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc017070 con 0x55e20d7dd720 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4ef1c00640 1 -- 172.21.15.32:0/423850081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc010730 con 0x55e20d7dd720 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4ef1c00640 1 -- 172.21.15.32:0/423850081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc006e60 con 0x55e20d7dd720 2024-04-12T19:51:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e20d87dc60 con 0x55e20d7dd720 2024-04-12T19:51:55.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.326+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20d87dfa0 con 0x55e20d7dd720 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.328+0000 7f4ef1c00640 1 -- 172.21.15.32:0/423850081 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4edc01a070 con 0x55e20d7dd720 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.329+0000 7f4ef1c00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4ed006b2d0 0x7f4ed006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.329+0000 7f4ef1c00640 1 -- 172.21.15.32:0/423850081 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4edc089d20 con 0x55e20d7dd720 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.329+0000 7f4ef4400640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4ed006b2d0 0x7f4ed006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.329+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55e20d883f10 0x55e20d91b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.329+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55e20d87d490 con 0x55e20d883f10 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.330+0000 7f4ef4400640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4ed006b2d0 0x7f4ed006d790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005630 tx=0x7f4ee40054c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.330+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55e20d883f10 0x55e20d91b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.331+0000 7f4ef4e00640 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55e20d883f10 0x55e20d91b850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.332+0000 7f4ef4e00640 1 -- 172.21.15.32:0/423850081 <== osd.7 v2:172.21.15.169:6822/883923800 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=19b] v0'0 uv10 ondisk = 0) ==== 164+0+19 (crc 0 0 0) 0x7f4eec05d040 con 0x55e20d883f10 2024-04-12T19:51:55.333 INFO:teuthology.orchestra.run.smithi032.stdout:alpha_head 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55e20d883f10 msgr2=0x55e20d91b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x55e20d883f10 0x55e20d91b850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4ed006b2d0 msgr2=0x7f4ed006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f4ed006b2d0 0x7f4ed006d790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005630 tx=0x7f4ee40054c0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 msgr2=0x55e20d87cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 --2- 172.21.15.32:0/423850081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e20d7dd720 0x55e20d87cf50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007dd0 tx=0x7f4edc006750 comp rx=0 tx=0).stop 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef4e00640 1 -- 172.21.15.32:0/423850081 reap_dead start 2024-04-12T19:51:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 shutdown_connections 2024-04-12T19:51:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 >> 172.21.15.32:0/423850081 conn(0x55e20d7d9ab0 msgr2=0x55e20d7d83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 shutdown_connections 2024-04-12T19:51:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.333+0000 7f4ef7492f40 1 -- 172.21.15.32:0/423850081 wait complete. 2024-04-12T19:51:55.338 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-04-12T19:51:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.444+0000 7fb14f5faf40 1 Processor -- start 2024-04-12T19:51:55.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.444+0000 7fb14f5faf40 1 -- start start 2024-04-12T19:51:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 0x556ef675add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b59f0 0x556ef673d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556ef66830b0 con 0x7fb130003630 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ef662bc90 con 0x556ef66b2020 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556ef6683350 con 0x556ef66b59f0 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41928/0 (socket says 172.21.15.32:41928) 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.445+0000 7fb14c600640 1 -- 172.21.15.32:0/2472832986 learned_addr learned my addr 172.21.15.32:0/2472832986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14bc00640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 0x556ef675add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14d000640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b59f0 0x556ef673d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 -- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b59f0 msgr2=0x556ef673d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b59f0 0x556ef673d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 -- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 msgr2=0x556ef675add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 0x556ef675add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 -- 172.21.15.32:0/2472832986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef66d1860 con 0x7fb130003630 2024-04-12T19:51:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14bc00640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 0x556ef675add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14c600640 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009800 tx=0x7fb13c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d349c2c151feb9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14a800640 1 -- 172.21.15.32:0/2472832986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0088d0 con 0x7fb130003630 2024-04-12T19:51:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14a800640 1 -- 172.21.15.32:0/2472832986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13c01a070 con 0x7fb130003630 2024-04-12T19:51:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.446+0000 7fb14a800640 1 -- 172.21.15.32:0/2472832986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c008c90 con 0x7fb130003630 2024-04-12T19:51:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 -- 172.21.15.32:0/2472832986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 msgr2=0x556ef675c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009800 tx=0x7fb13c005a00 comp rx=0 tx=0).stop 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 -- 172.21.15.32:0/2472832986 shutdown_connections 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b59f0 0x556ef673d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b2020 0x556ef675add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/2472832986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef675c600 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 -- 172.21.15.32:0/2472832986 >> 172.21.15.32:0/2472832986 conn(0x556ef66b3ab0 msgr2=0x556ef66d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 -- 172.21.15.32:0/2472832986 shutdown_connections 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.447+0000 7fb14f5faf40 1 -- 172.21.15.32:0/2472832986 wait complete. 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.448+0000 7fb14f5faf40 1 Processor -- start 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.448+0000 7fb14f5faf40 1 -- start start 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b2020 0x556ef67fa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b59f0 0x556ef6755180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef66d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556ef675eb50 con 0x7fb130003630 2024-04-12T19:51:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ef66b53f0 con 0x556ef66b59f0 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14f5faf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556ef67eb2b0 con 0x556ef66b2020 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.449+0000 7fb14d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef66d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef66d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41942/0 (socket says 172.21.15.32:41942) 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 -- 172.21.15.32:0/3312625768 learned_addr learned my addr 172.21.15.32:0/3312625768 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14c600640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b2020 0x556ef67fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14bc00640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b59f0 0x556ef6755180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 -- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b2020 msgr2=0x556ef67fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ef66b2020 0x556ef67fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 -- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b59f0 msgr2=0x556ef6755180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b59f0 0x556ef6755180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 -- 172.21.15.32:0/3312625768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef6740e70 con 0x7fb130003630 2024-04-12T19:51:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14bc00640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ef66b59f0 0x556ef6755180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.450+0000 7fb14d000640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef66d0090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb144044cf0 tx=0x7fb144005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.451+0000 7fb149400640 1 -- 172.21.15.32:0/3312625768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144005a80 con 0x7fb130003630 2024-04-12T19:51:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.451+0000 7fb149400640 1 -- 172.21.15.32:0/3312625768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb144004820 con 0x7fb130003630 2024-04-12T19:51:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.451+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ef67ea6a0 con 0x7fb130003630 2024-04-12T19:51:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.451+0000 7fb149400640 1 -- 172.21.15.32:0/3312625768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144004c10 con 0x7fb130003630 2024-04-12T19:51:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.451+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef67eac60 con 0x7fb130003630 2024-04-12T19:51:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.453+0000 7fb149400640 1 -- 172.21.15.32:0/3312625768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb144052470 con 0x7fb130003630 2024-04-12T19:51:55.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.454+0000 7fb149400640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb12c06b2d0 0x7fb12c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.454+0000 7fb149400640 1 -- 172.21.15.32:0/3312625768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb1440cd450 con 0x7fb130003630 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.455+0000 7fb14c600640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb12c06b2d0 0x7fb12c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.455+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x556ef66d05d0 0x556ef67eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.455+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- osd_op(unknown.0.0:1 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x556ef67556c0 con 0x556ef66d05d0 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.455+0000 7fb14bc00640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x556ef66d05d0 0x556ef67eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.455+0000 7fb14c600640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb12c06b2d0 0x7fb12c06d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb13c017040 tx=0x7fb13c014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14bc00640 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x556ef66d05d0 0x556ef67eb740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14bc00640 1 -- 172.21.15.32:0/3312625768 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=19b] v0'0 uv6 ondisk = 0) ==== 154+0+19 (crc 0 0 0) 0x7fb13800b040 con 0x556ef66d05d0 2024-04-12T19:51:55.456 INFO:teuthology.orchestra.run.smithi032.stdout:mydir_head 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x556ef66d05d0 msgr2=0x556ef67eb740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x556ef66d05d0 0x556ef67eb740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb12c06b2d0 msgr2=0x7fb12c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb12c06b2d0 0x7fb12c06d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb13c017040 tx=0x7fb13c014000 comp rx=0 tx=0).stop 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 msgr2=0x556ef66d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14f5faf40 1 --2- 172.21.15.32:0/3312625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb130003630 0x556ef66d0090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb144044cf0 tx=0x7fb144005d80 comp rx=0 tx=0).stop 2024-04-12T19:51:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.456+0000 7fb14d000640 1 -- 172.21.15.32:0/3312625768 reap_dead start 2024-04-12T19:51:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.457+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 shutdown_connections 2024-04-12T19:51:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.457+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 >> 172.21.15.32:0/3312625768 conn(0x556ef66b3ab0 msgr2=0x556ef66b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.457+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 shutdown_connections 2024-04-12T19:51:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.457+0000 7fb14f5faf40 1 -- 172.21.15.32:0/3312625768 wait complete. 2024-04-12T19:51:55.461 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-12T19:51:55.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.569+0000 7f124ba0bf40 1 Processor -- start 2024-04-12T19:51:55.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.569+0000 7f124ba0bf40 1 -- start start 2024-04-12T19:51:55.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b034ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b03412370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b0342e190 con 0x555b0349f210 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555b0338ac90 con 0x7f122c003630 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x555b033e4f00 con 0x555b0349f630 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f123f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f123f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41956/0 (socket says 172.21.15.32:41956) 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f123f400640 1 -- 172.21.15.32:0/3890858077 learned_addr learned my addr 172.21.15.32:0/3890858077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f1247e00640 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b034ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f1248800640 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b03412370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.570+0000 7f123f400640 1 -- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 msgr2=0x555b03412370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f123f400640 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b03412370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f123f400640 1 -- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 msgr2=0x555b034ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f123f400640 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b034ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f123f400640 1 -- 172.21.15.32:0/3890858077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b03412ae0 con 0x555b0349f210 2024-04-12T19:51:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f123f400640 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37b72a6ed5d0c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f1246a00640 1 -- 172.21.15.32:0/3890858077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123000ec20 con 0x555b0349f210 2024-04-12T19:51:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f1246a00640 1 -- 172.21.15.32:0/3890858077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230013070 con 0x555b0349f210 2024-04-12T19:51:55.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.571+0000 7f1246a00640 1 -- 172.21.15.32:0/3890858077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230018470 con 0x555b0349f210 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3890858077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 msgr2=0x555b03411e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123000ce70 comp rx=0 tx=0).stop 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3890858077 shutdown_connections 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b03412370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b03411e30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3890858077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b034ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3890858077 >> 172.21.15.32:0/3890858077 conn(0x555b0340e760 msgr2=0x555b034b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3890858077 shutdown_connections 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.572+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3890858077 wait complete. 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.573+0000 7f124ba0bf40 1 Processor -- start 2024-04-12T19:51:55.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.573+0000 7f124ba0bf40 1 -- start start 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b0349e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b035516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b03411c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555b034bab70 con 0x555b0349f210 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555b034b9380 con 0x7f122c003630 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f124ba0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x555b034b9660 con 0x555b0349f630 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f1247e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b0349e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f1247e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b0349e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41958/0 (socket says 172.21.15.32:41958) 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.574+0000 7f1247e00640 1 -- 172.21.15.32:0/3982737413 learned_addr learned my addr 172.21.15.32:0/3982737413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f123f400640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b035516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 -- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 msgr2=0x555b035516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555b0349f630 0x555b035516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 -- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 msgr2=0x555b03411c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1248800640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b03411c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f122c003630 0x555b03411c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 -- 172.21.15.32:0/3982737413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0342baa0 con 0x555b0349f210 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1247e00640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b0349e070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1238009f30 tx=0x7f12380074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1245600640 1 -- 172.21.15.32:0/3982737413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238006790 con 0x555b0349f210 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f1245600640 1 -- 172.21.15.32:0/3982737413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238006930 con 0x555b0349f210 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.575+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b0342bd80 con 0x555b0349f210 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.576+0000 7f1245600640 1 -- 172.21.15.32:0/3982737413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238006d20 con 0x555b0349f210 2024-04-12T19:51:55.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.576+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b03425970 con 0x555b0349f210 2024-04-12T19:51:55.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.578+0000 7f1245600640 1 -- 172.21.15.32:0/3982737413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1238016440 con 0x555b0349f210 2024-04-12T19:51:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.578+0000 7f1245600640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f122406b2d0 0x7f122406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.579+0000 7f1245600640 1 -- 172.21.15.32:0/3982737413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f123808b0b0 con 0x555b0349f210 2024-04-12T19:51:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.579+0000 7f123f400640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f122406b2d0 0x7f122406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.579+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x555b03551be0 0x555b03425e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.579+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x555b033e40d0 con 0x555b03551be0 2024-04-12T19:51:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.579+0000 7f1248800640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x555b03551be0 0x555b03425e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f123f400640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f122406b2d0 0x7f122406d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123001a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f1248800640 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x555b03551be0 0x555b03425e90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f1248800640 1 -- 172.21.15.32:0/3982737413 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (92) out=92b] v0'0 uv6 ondisk = 0) ==== 164+0+92 (crc 0 0 0) 0x7f124004f540 con 0x555b03551be0 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x555b03551be0 msgr2=0x555b03425e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x555b03551be0 0x555b03425e90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.580+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f122406b2d0 msgr2=0x7f122406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f122406b2d0 0x7f122406d790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f123000c9b0 tx=0x7f123001a040 comp rx=0 tx=0).stop 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 msgr2=0x555b0349e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 --2- 172.21.15.32:0/3982737413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555b0349f210 0x555b0349e070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1238009f30 tx=0x7f12380074b0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f1248800640 1 -- 172.21.15.32:0/3982737413 reap_dead start 2024-04-12T19:51:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 shutdown_connections 2024-04-12T19:51:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 >> 172.21.15.32:0/3982737413 conn(0x555b0340e760 msgr2=0x555b033e39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 shutdown_connections 2024-04-12T19:51:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.581+0000 7f124ba0bf40 1 -- 172.21.15.32:0/3982737413 wait complete. 2024-04-12T19:51:55.586 INFO:teuthology.orchestra.run.smithi032.stdout:Valphamydir 2024-04-12T19:51:55.586 DEBUG:teuthology.orchestra.run.smithi032:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: "ancestors": [ 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1099511627776, 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "alpha", 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 6 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-12T19:51:55.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1, 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "mydir", 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 8 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 5, 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: "old_pools": [] 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:51:55.712 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:55.723 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary 2024-04-12T19:51:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.832+0000 7fefd0a10280 1 -- start start 2024-04-12T19:51:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.832+0000 7fefd0a10280 1 Processor -- start 2024-04-12T19:51:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff6045b800 0x55ff61222580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff6045bc00 0x55ff61222b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff611c8600 con 0x55ff6045ac00 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff611c8480 con 0x55ff6045b800 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ff611c8300 con 0x55ff6045bc00 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.833+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41978/0 (socket says 172.21.15.32:41978) 2024-04-12T19:51:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 -- 172.21.15.32:0/2727651608 learned_addr learned my addr 172.21.15.32:0/2727651608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcd000640 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff6045b800 0x55ff61222580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 -- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff6045bc00 msgr2=0x55ff61222b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff6045bc00 0x55ff61222b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 -- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff6045b800 msgr2=0x55ff61222580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff6045b800 0x55ff61222580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 -- 172.21.15.32:0/2727651608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff60490780 con 0x55ff6045ac00 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefcda00640 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55ff61277440 tx=0x55ff611ec8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a92b0ff3efc5274 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.834+0000 7fefca800640 1 -- 172.21.15.32:0/2727651608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048e4e0 con 0x55ff6045ac00 2024-04-12T19:51:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefca800640 1 -- 172.21.15.32:0/2727651608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ff60486e00 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefca800640 1 -- 172.21.15.32:0/2727651608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048eb60 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff604903c0 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ff60491680 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 4 waiting for MDS map... 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefca800640 1 -- 172.21.15.32:0/2727651608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x55ff6131e600 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefca800640 1 -- 172.21.15.32:0/2727651608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x55ff604b7180 con 0x55ff6045ac00 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 4 Got MDS map 16 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 10 main: JournalTool::main 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.835+0000 7fefd0a10280 4 main: JournalTool: connecting to RADOS... 2024-04-12T19:51:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.836+0000 7fefd0a10280 1 Processor -- start 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.836+0000 7fefd0a10280 1 -- start start 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff611c8180 con 0x55ff61321800 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff60481800 con 0x55ff61321c00 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ff60480a80 con 0x55ff61321400 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.837+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41982/0 (socket says 172.21.15.32:41982) 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 -- 172.21.15.32:0/2263013760 learned_addr learned my addr 172.21.15.32:0/2263013760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefce400640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 -- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 msgr2=0x55ff61223080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcd000640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 -- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 msgr2=0x55ff61223b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 -- 172.21.15.32:0/2263013760 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff604914a0 con 0x55ff61321800 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefce400640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.838+0000 7fefcda00640 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55ff61338660 tx=0x55ff611ec960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4862e16805eeb0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefc8000640 1 -- 172.21.15.32:0/2263013760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048e340 con 0x55ff61321800 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefc8000640 1 -- 172.21.15.32:0/2263013760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ff60486c40 con 0x55ff61321800 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefc8000640 1 -- 172.21.15.32:0/2263013760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048e4e0 con 0x55ff61321800 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 -- 172.21.15.32:0/2263013760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 msgr2=0x55ff61223600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55ff61338660 tx=0x55ff611ec960 comp rx=0 tx=0).stop 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 -- 172.21.15.32:0/2263013760 shutdown_connections 2024-04-12T19:51:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2263013760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 -- 172.21.15.32:0/2263013760 >> 172.21.15.32:0/2263013760 conn(0x55ff61321000 msgr2=0x55ff60540800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 -- 172.21.15.32:0/2263013760 shutdown_connections 2024-04-12T19:51:55.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.839+0000 7fefd0a10280 1 -- 172.21.15.32:0/2263013760 wait complete. 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.840+0000 7fefd0a10280 1 Processor -- start 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.840+0000 7fefd0a10280 1 -- start start 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff611c9800 con 0x55ff61321800 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ff611c9980 con 0x55ff61321c00 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefd0a10280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ff611c8d80 con 0x55ff61321400 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.841+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41998/0 (socket says 172.21.15.32:41998) 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 -- 172.21.15.32:0/1981083228 learned_addr learned my addr 172.21.15.32:0/1981083228 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefce400640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcd000640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 -- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 msgr2=0x55ff61223080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 -- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 msgr2=0x55ff61223b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 -- 172.21.15.32:0/1981083228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff60490780 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcd000640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefcda00640 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55ff61339320 tx=0x55ff6133ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.842+0000 7fefc6200640 1 -- 172.21.15.32:0/1981083228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048eb60 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefc6200640 1 -- 172.21.15.32:0/1981083228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ff604861c0 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefc6200640 1 -- 172.21.15.32:0/1981083228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff6048e000 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff604903c0 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff60491680 con 0x55ff61321800 2024-04-12T19:51:55.847 INFO:teuthology.orchestra.run.smithi032.stderr:Error (2024-04-12T19:51:55.843+0000 7fefd0a10280 -1 main: Cannot run cephfs-journal-tool on an active file system! 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:(1) Operation not permitted) 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 msgr2=0x55ff61223600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55ff61339320 tx=0x55ff6133ef00 comp rx=0 tx=0).stop 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 shutdown_connections 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff61321c00 0x55ff61223b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff61321800 0x55ff61223600 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 --2- 172.21.15.32:0/1981083228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff61321400 0x55ff61223080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 >> 172.21.15.32:0/1981083228 conn(0x55ff61321000 msgr2=0x55ff60540800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 shutdown_connections 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.843+0000 7fefd0a10280 1 -- 172.21.15.32:0/1981083228 wait complete. 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 msgr2=0x55ff61222000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55ff61277440 tx=0x55ff611ec8a0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 shutdown_connections 2024-04-12T19:51:55.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff6045bc00 0x55ff61222b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff6045b800 0x55ff61222580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 --2- 172.21.15.32:0/2727651608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff6045ac00 0x55ff61222000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 >> 172.21.15.32:0/2727651608 conn(0x55ff6045b000 msgr2=0x55ff60541000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 shutdown_connections 2024-04-12T19:51:55.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:55.844+0000 7fefd0a10280 1 -- 172.21.15.32:0/2727651608 wait complete. 2024-04-12T19:51:55.849 DEBUG:teuthology.orchestra.run:got remote process result: 255 2024-04-12T19:51:55.850 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-12T19:51:55.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-12T19:51:55.960 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-12T19:51:56.081 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-12T19:51:56.081 INFO:tasks.cephfs.mount:Terminating background process 2024-04-12T19:51:56.081 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:56.081 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:56.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053546,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-12T19:51:56.141 INFO:tasks.cephfs.mount:Terminating background process 2024-04-12T19:51:56.141 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:56.141 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:56.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-12T19:51:56.199 INFO:tasks.cephfs.mount:Terminating background process 2024-04-12T19:51:56.200 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:51:56.200 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:56.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053546,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-12T19:51:56.261 INFO:tasks.cephfs.mount:Terminating background process 2024-04-12T19:51:56.261 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:51:56.261 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:51:56.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-12T19:51:56.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:56.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.540+0000 7f210d200640 1 Processor -- start 2024-04-12T19:51:56.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.541+0000 7f210d200640 1 -- start start 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.542+0000 7f210d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21080b5a30 0x7f21080af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f210d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f21080af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f210d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21080b1860 con 0x7f210815ea70 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f210d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f21080b1b40 con 0x7f21080b5a30 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f210d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21080b1e20 con 0x7f21080b12e0 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21080b5a30 0x7f21080af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21080b5a30 0x7f21080af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42006/0 (socket says 172.21.15.32:42006) 2024-04-12T19:51:56.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 -- 172.21.15.32:0/128361788 learned_addr learned my addr 172.21.15.32:0/128361788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2106800640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f21080af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 -- 172.21.15.32:0/128361788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 msgr2=0x7f21080af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f21080af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 -- 172.21.15.32:0/128361788 >> v1:172.21.15.169:6789/0 conn(0x7f210815ea70 legacy=0x7f21080b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.543+0000 7f2105400640 1 -- 172.21.15.32:0/128361788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21080b2100 con 0x7f21080b5a30 2024-04-12T19:51:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.545+0000 7f2105400640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21080b5a30 0x7f21080af270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009870 tx=0x7f20f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6186ee7c406e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:56.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.546+0000 7f2104a00640 1 -- 172.21.15.32:0/128361788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4035020 con 0x7f21080b5a30 2024-04-12T19:51:56.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.546+0000 7f2104a00640 1 -- 172.21.15.32:0/128361788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f4031640 con 0x7f21080b5a30 2024-04-12T19:51:56.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.546+0000 7f2104a00640 1 -- 172.21.15.32:0/128361788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f402d040 con 0x7f21080b5a30 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.546+0000 7f210d200640 1 -- 172.21.15.32:0/128361788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21080b5a30 msgr2=0x7f21080af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.546+0000 7f210d200640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21080b5a30 0x7f21080af270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009870 tx=0x7f20f402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 -- 172.21.15.32:0/128361788 shutdown_connections 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f21080af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 --2- 172.21.15.32:0/128361788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21080b5a30 0x7f21080af270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 -- 172.21.15.32:0/128361788 >> 172.21.15.32:0/128361788 conn(0x7f21080b6f50 msgr2=0x7f210809aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 -- 172.21.15.32:0/128361788 shutdown_connections 2024-04-12T19:51:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.547+0000 7f210d200640 1 -- 172.21.15.32:0/128361788 wait complete. 2024-04-12T19:51:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.549+0000 7f210d200640 1 Processor -- start 2024-04-12T19:51:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.549+0000 7f210d200640 1 -- start start 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.550+0000 7f210d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f2108156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f210d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21080b5a30 0x7f2108141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f210d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f210d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f210815efd0 con 0x7f210815ea70 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f210d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f210815ee50 con 0x7f21080b5a30 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f210d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f210809b2a0 con 0x7f21080b12e0 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42014/0 (socket says 172.21.15.32:42014) 2024-04-12T19:51:56.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 -- 172.21.15.32:0/909814150 learned_addr learned my addr 172.21.15.32:0/909814150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2105e00640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f2108156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 -- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 msgr2=0x7f2108156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f2108156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 -- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21080b5a30 msgr2=0x7f2108141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2105400640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21080b5a30 0x7f2108141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21080b5a30 0x7f2108141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.551+0000 7f2106800640 1 -- 172.21.15.32:0/909814150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2108153390 con 0x7f210815ea70 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f2106800640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2100007e40 tx=0x7f210000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100014620 con 0x7f210815ea70 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2100019070 con 0x7f210815ea70 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100014a20 con 0x7f210815ea70 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081f4e30 con 0x7f210815ea70 2024-04-12T19:51:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.552+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081f53f0 con 0x7f210815ea70 2024-04-12T19:51:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.554+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21081f4c60 con 0x7f210815ea70 2024-04-12T19:51:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.555+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2100014bc0 con 0x7f210815ea70 2024-04-12T19:51:56.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.555+0000 7f20fe000640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 0x7f20e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:56.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.556+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f210008c410 con 0x7f210815ea70 2024-04-12T19:51:56.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.556+0000 7f2105e00640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 0x7f20e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:56.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.557+0000 7f2105e00640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 0x7f20e806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006fd0 tx=0x7f20f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.560+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f210005a840 con 0x7f210815ea70 2024-04-12T19:51:56.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.808+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2108068c80 con 0x7f210815ea70 2024-04-12T19:51:56.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.809+0000 7f20fe000640 1 -- 172.21.15.32:0/909814150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f210005a140 con 0x7f210815ea70 2024-04-12T19:51:56.809 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:56.809 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:51:56.809 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.811+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 msgr2=0x7f20e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.811+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 0x7f20e806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006fd0 tx=0x7f20f8008040 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.811+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 msgr2=0x7f2108152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.811+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2100007e40 tx=0x7f210000e4b0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 shutdown_connections 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f20e806c8f0 0x7f20e806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f210815ea70 0x7f2108152ad0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21080b5a30 0x7f2108141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 --2- 172.21.15.32:0/909814150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21080b12e0 0x7f2108156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:56.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 >> 172.21.15.32:0/909814150 conn(0x7f21080b6f50 msgr2=0x7f210815dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:56.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 shutdown_connections 2024-04-12T19:51:56.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:56.812+0000 7f210d200640 1 -- 172.21.15.32:0/909814150 wait complete. 2024-04-12T19:51:56.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.056+0000 7f0edde00640 1 Processor -- start 2024-04-12T19:51:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.056+0000 7f0edde00640 1 -- start start 2024-04-12T19:51:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0edde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0edde00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ed80b3470 0x7f0ed80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0edde00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed80d2d90 con 0x7f0ed815eac0 2024-04-12T19:51:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0edde00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed80d1cd0 con 0x7f0ed80b3470 2024-04-12T19:51:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0edde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed80d3070 con 0x7f0ed80d1300 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0ed7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ed80b3470 0x7f0ed80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0ed7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ed80b3470 0x7f0ed80b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42018/0 (socket says 172.21.15.32:42018) 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.057+0000 7f0ed7400640 1 -- 172.21.15.32:0/1577074396 learned_addr learned my addr 172.21.15.32:0/1577074396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.058+0000 7f0ed6000640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.058+0000 7f0ed7400640 1 -- 172.21.15.32:0/1577074396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 msgr2=0x7f0ed80d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.058+0000 7f0ed7400640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.058+0000 7f0ed7400640 1 -- 172.21.15.32:0/1577074396 >> v1:172.21.15.169:6789/0 conn(0x7f0ed815eac0 legacy=0x7f0ed80d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.058+0000 7f0ed7400640 1 -- 172.21.15.32:0/1577074396 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed80d3350 con 0x7f0ed80b3470 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.059+0000 7f0ed7400640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80b3470 0x7f0ed80b38e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009870 tx=0x7f0ed002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be78ae08cff64ab server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.059+0000 7f0ed5600640 1 -- 172.21.15.32:0/1577074396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0035020 con 0x7f0ed80b3470 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.059+0000 7f0ed5600640 1 -- 172.21.15.32:0/1577074396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed0031640 con 0x7f0ed80b3470 2024-04-12T19:51:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.059+0000 7f0ed5600640 1 -- 172.21.15.32:0/1577074396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002d040 con 0x7f0ed80b3470 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 -- 172.21.15.32:0/1577074396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80b3470 msgr2=0x7f0ed80b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80b3470 0x7f0ed80b38e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009870 tx=0x7f0ed002b8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 -- 172.21.15.32:0/1577074396 shutdown_connections 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80b3470 0x7f0ed80b38e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 --2- 172.21.15.32:0/1577074396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 -- 172.21.15.32:0/1577074396 >> 172.21.15.32:0/1577074396 conn(0x7f0ed8073690 msgr2=0x7f0ed806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 -- 172.21.15.32:0/1577074396 shutdown_connections 2024-04-12T19:51:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.060+0000 7f0edde00640 1 -- 172.21.15.32:0/1577074396 wait complete. 2024-04-12T19:51:57.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.062+0000 7f0edde00640 1 Processor -- start 2024-04-12T19:51:57.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.062+0000 7f0edde00640 1 -- start start 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 0x7f0ed81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed815eea0 con 0x7f0ed815eac0 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed80d3910 con 0x7f0ed80b3470 2024-04-12T19:51:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0edde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed80d4590 con 0x7f0ed80d1300 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0ed7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.063+0000 7f0ed7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42030/0 (socket says 172.21.15.32:42030) 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 -- 172.21.15.32:0/1296879930 learned_addr learned my addr 172.21.15.32:0/1296879930 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed6000640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed6a00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 0x7f0ed81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 -- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 msgr2=0x7f0ed80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 -- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 msgr2=0x7f0ed81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 0x7f0ed81595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 -- 172.21.15.32:0/1296879930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed80cdbd0 con 0x7f0ed815eac0 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed6a00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 0x7f0ed81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.064+0000 7f0ed7400640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009870 tx=0x7f0ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.065+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0040070 con 0x7f0ed815eac0 2024-04-12T19:51:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.065+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed0039c10 con 0x7f0ed815eac0 2024-04-12T19:51:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.065+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0035040 con 0x7f0ed815eac0 2024-04-12T19:51:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.065+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed8159cb0 con 0x7f0ed815eac0 2024-04-12T19:51:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.065+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed815a270 con 0x7f0ed815eac0 2024-04-12T19:51:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.066+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed8068c80 con 0x7f0ed815eac0 2024-04-12T19:51:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.067+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ed004b060 con 0x7f0ed815eac0 2024-04-12T19:51:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.068+0000 7f0ecf400640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.068+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0ed00ad630 con 0x7f0ed815eac0 2024-04-12T19:51:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.068+0000 7f0ed6a00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.069+0000 7f0ed6a00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800a830 tx=0x7f0ec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.072+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ed002f300 con 0x7f0ed815eac0 2024-04-12T19:51:57.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.282+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ed80d1830 con 0x7f0ed815eac0 2024-04-12T19:51:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.284+0000 7f0ecf400640 1 -- 172.21.15.32:0/1296879930 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f0ed007ba50 con 0x7f0ed815eac0 2024-04-12T19:51:57.285 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:57.285 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:38.305869+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 msgr2=0x7f0eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800a830 tx=0x7f0ec8008040 comp rx=0 tx=0).stop 2024-04-12T19:51:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 msgr2=0x7f0ed8154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009870 tx=0x7f0ed00046c0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 shutdown_connections 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed815eac0 0x7f0ed8154e70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ed80d1300 0x7f0ed80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 --2- 172.21.15.32:0/1296879930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ed80b3470 0x7f0ed81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 >> 172.21.15.32:0/1296879930 conn(0x7f0ed8073690 msgr2=0x7f0ed815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 shutdown_connections 2024-04-12T19:51:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.287+0000 7f0edde00640 1 -- 172.21.15.32:0/1296879930 wait complete. 2024-04-12T19:51:57.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T19:50:11.662919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-12T19:51:34.775305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-12T19:51:35.764856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T19:51:57.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T19:51:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.533+0000 7f2023200640 1 Processor -- start 2024-04-12T19:51:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.534+0000 7f2023200640 1 -- start start 2024-04-12T19:51:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202415e960 0x7f20240ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202415ed40 0x7f20240cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2024162670 0x7f20240cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f20240ba740 con 0x7f202415ed40 2024-04-12T19:51:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2024162c30 con 0x7f2024162670 2024-04-12T19:51:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f2023200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2024162f10 con 0x7f202415e960 2024-04-12T19:51:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f201be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202415ed40 0x7f20240cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f201be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f202415ed40 0x7f20240cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42044/0 (socket says 172.21.15.32:42044) 2024-04-12T19:51:57.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.535+0000 7f201be00640 1 -- 172.21.15.32:0/1744561947 learned_addr learned my addr 172.21.15.32:0/1744561947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f2022800640 1 --2- 172.21.15.32:0/1744561947 >> v2:172.21.15.169:3300/0 conn(0x7f2024162670 0x7f20240cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f2021e00640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202415e960 0x7f20240ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f201be00640 1 -- 172.21.15.32:0/1744561947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202415e960 msgr2=0x7f20240ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f201be00640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202415e960 0x7f20240ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f201be00640 1 -- 172.21.15.32:0/1744561947 >> v2:172.21.15.169:3300/0 conn(0x7f2024162670 msgr2=0x7f20240cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f201be00640 1 --2- 172.21.15.32:0/1744561947 >> v2:172.21.15.169:3300/0 conn(0x7f2024162670 0x7f20240cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.536+0000 7f201be00640 1 -- 172.21.15.32:0/1744561947 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20241631f0 con 0x7f202415ed40 2024-04-12T19:51:57.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.537+0000 7f201be00640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415ed40 0x7f20240cf5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f20240d5890 tx=0x7f200c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffed93608d1b15b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.537+0000 7f2021400640 1 -- 172.21.15.32:0/1744561947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0317d0 con 0x7f202415ed40 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.537+0000 7f2021400640 1 -- 172.21.15.32:0/1744561947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c037070 con 0x7f202415ed40 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.537+0000 7f2021400640 1 -- 172.21.15.32:0/1744561947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c031b90 con 0x7f202415ed40 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 -- 172.21.15.32:0/1744561947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415ed40 msgr2=0x7f20240cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415ed40 0x7f20240cf5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f20240d5890 tx=0x7f200c02b870 comp rx=0 tx=0).stop 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 -- 172.21.15.32:0/1744561947 shutdown_connections 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415ed40 0x7f20240cf5d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 --2- 172.21.15.32:0/1744561947 >> v2:172.21.15.169:3300/0 conn(0x7f2024162670 0x7f20240cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 --2- 172.21.15.32:0/1744561947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202415e960 0x7f20240ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 -- 172.21.15.32:0/1744561947 >> 172.21.15.32:0/1744561947 conn(0x7f202406cf90 msgr2=0x7f202406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 -- 172.21.15.32:0/1744561947 shutdown_connections 2024-04-12T19:51:57.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.538+0000 7f2023200640 1 -- 172.21.15.32:0/1744561947 wait complete. 2024-04-12T19:51:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.540+0000 7f2023200640 1 Processor -- start 2024-04-12T19:51:57.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.540+0000 7f2023200640 1 -- start start 2024-04-12T19:51:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 0x7f202415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2024162670 0x7f20240cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b6400 con 0x7f202415e960 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b6280 con 0x7f202415ed40 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2023200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20240b6580 con 0x7f2024162670 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.541+0000 7f2021e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42050/0 (socket says 172.21.15.32:42050) 2024-04-12T19:51:57.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 -- 172.21.15.32:0/3164649963 learned_addr learned my addr 172.21.15.32:0/3164649963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f201be00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 0x7f202415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 -- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2024162670 msgr2=0x7f20240cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2024162670 0x7f20240cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 -- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 msgr2=0x7f202415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 0x7f202415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 -- 172.21.15.32:0/3164649963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20240ce510 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f201be00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 0x7f202415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.542+0000 7f2021e00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f201400c7f0 tx=0x7f201400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.543+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014015070 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.543+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2014004220 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.543+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014007620 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.543+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2024160b00 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.543+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241610c0 con 0x7f202415e960 2024-04-12T19:51:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.544+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240755c0 con 0x7f202415e960 2024-04-12T19:51:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.545+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20140079d0 con 0x7f202415e960 2024-04-12T19:51:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.546+0000 7f201a000640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.546+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f201408bbe0 con 0x7f202415e960 2024-04-12T19:51:57.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.546+0000 7f201be00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.547+0000 7f201be00640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 0x7f200006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f200c036000 tx=0x7f200c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:57.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.551+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2014059f80 con 0x7f202415e960 2024-04-12T19:51:57.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.758+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20240d2510 con 0x7f202415e960 2024-04-12T19:51:57.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.759+0000 7f201a000640 1 -- 172.21.15.32:0/3164649963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f2014059880 con 0x7f202415e960 2024-04-12T19:51:57.759 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 msgr2=0x7f200006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 0x7f200006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f200c036000 tx=0x7f200c008030 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 msgr2=0x7f20240cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f201400c7f0 tx=0x7f201400ccb0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 shutdown_connections 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.762+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f200006c8f0 0x7f200006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2024162670 0x7f20240cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202415ed40 0x7f202415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 --2- 172.21.15.32:0/3164649963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f202415e960 0x7f20240cdee0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:57.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 >> 172.21.15.32:0/3164649963 conn(0x7f202406cf90 msgr2=0x7f20240d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:57.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 shutdown_connections 2024-04-12T19:51:57.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:57.763+0000 7f2023200640 1 -- 172.21.15.32:0/3164649963 wait complete. 2024-04-12T19:51:57.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-12T19:51:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.008+0000 7f1ab3e00640 1 Processor -- start 2024-04-12T19:51:58.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.008+0000 7f1ab3e00640 1 -- start start 2024-04-12T19:51:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.010+0000 7f1ab3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ab41461c0 0x7f1ab4071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.010+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab40cebb0 0x7f1ab406df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.010+0000 7f1ab3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab40cd2d0 con 0x7f1ab4072420 2024-04-12T19:51:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.010+0000 7f1ab3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab40cf1d0 con 0x7f1ab41461c0 2024-04-12T19:51:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.010+0000 7f1ab3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab4072880 con 0x7f1ab40cebb0 2024-04-12T19:51:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ab41461c0 0x7f1ab4071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab40cebb0 0x7f1ab406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ab41461c0 0x7f1ab4071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:33130/0 (socket says 172.21.15.32:33130) 2024-04-12T19:51:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 -- 172.21.15.32:0/4234050920 learned_addr learned my addr 172.21.15.32:0/4234050920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 -- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab40cebb0 msgr2=0x7f1ab406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 --2- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab40cebb0 0x7f1ab406df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 -- 172.21.15.32:0/4234050920 >> v1:172.21.15.169:6789/0 conn(0x7f1ab4072420 legacy=0x7f1ab40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.011+0000 7f1ab2a00640 1 -- 172.21.15.32:0/4234050920 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40cd5b0 con 0x7f1ab41461c0 2024-04-12T19:51:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.013+0000 7f1ab2a00640 1 --2- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab41461c0 0x7f1ab4071ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009870 tx=0x7f1aa402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd4644b22b86519 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.013+0000 7f1ab1600640 1 -- 172.21.15.32:0/4234050920 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4034070 con 0x7f1ab41461c0 2024-04-12T19:51:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.013+0000 7f1ab1600640 1 -- 172.21.15.32:0/4234050920 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4030840 con 0x7f1ab41461c0 2024-04-12T19:51:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.013+0000 7f1ab1600640 1 -- 172.21.15.32:0/4234050920 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4030c10 con 0x7f1ab41461c0 2024-04-12T19:51:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 -- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab41461c0 msgr2=0x7f1ab4071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab41461c0 0x7f1ab4071ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009870 tx=0x7f1aa402b980 comp rx=0 tx=0).stop 2024-04-12T19:51:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 -- 172.21.15.32:0/4234050920 shutdown_connections 2024-04-12T19:51:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab40cebb0 0x7f1ab406df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/4234050920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab41461c0 0x7f1ab4071ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 -- 172.21.15.32:0/4234050920 >> 172.21.15.32:0/4234050920 conn(0x7f1ab406b930 msgr2=0x7f1ab406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 -- 172.21.15.32:0/4234050920 shutdown_connections 2024-04-12T19:51:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.014+0000 7f1ab3e00640 1 -- 172.21.15.32:0/4234050920 wait complete. 2024-04-12T19:51:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.016+0000 7f1ab3e00640 1 Processor -- start 2024-04-12T19:51:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.016+0000 7f1ab3e00640 1 -- start start 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab4072420 0x7f1ab413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab40cebb0 0x7f1ab40c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40cdb70 con 0x7f1ab41461c0 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40cd080 con 0x7f1ab40cebb0 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab406e460 con 0x7f1ab4072420 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42062/0 (socket says 172.21.15.32:42062) 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.017+0000 7f1ab3400640 1 -- 172.21.15.32:0/660436272 learned_addr learned my addr 172.21.15.32:0/660436272 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab2a00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab4072420 0x7f1ab413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab2000640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab40cebb0 0x7f1ab40c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 -- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab4072420 msgr2=0x7f1ab413efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab4072420 0x7f1ab413efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 -- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab40cebb0 msgr2=0x7f1ab40c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab40cebb0 0x7f1ab40c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 -- 172.21.15.32:0/660436272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40ab630 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.018+0000 7f1ab3400640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dd0 tx=0x7f1aac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.019+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac018020 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.019+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac014480 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.019+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac014870 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.019+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab40ab910 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.019+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab40abed0 con 0x7f1ab41461c0 2024-04-12T19:51:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.020+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab40755c0 con 0x7f1ab41461c0 2024-04-12T19:51:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.021+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aac005d10 con 0x7f1ab41461c0 2024-04-12T19:51:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.022+0000 7f1aaaa00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.022+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1aac08c6d0 con 0x7f1ab41461c0 2024-04-12T19:51:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.022+0000 7f1ab2a00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.023+0000 7f1ab2a00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4036000 tx=0x7f1aa403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.027+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1aac022070 con 0x7f1ab41461c0 2024-04-12T19:51:58.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.230+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ab406c600 con 0x7f1ab41461c0 2024-04-12T19:51:58.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.231+0000 7f1aaaa00640 1 -- 172.21.15.32:0/660436272 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f1aac010040 con 0x7f1ab41461c0 2024-04-12T19:51:58.232 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-12T19:51:58.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 msgr2=0x7f1a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4036000 tx=0x7f1aa403b040 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 msgr2=0x7f1ab40c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dd0 tx=0x7f1aac00c490 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 shutdown_connections 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ab41461c0 0x7f1ab40c6150 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab40cebb0 0x7f1ab40c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 --2- 172.21.15.32:0/660436272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab4072420 0x7f1ab413efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 >> 172.21.15.32:0/660436272 conn(0x7f1ab406b930 msgr2=0x7f1ab406f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 shutdown_connections 2024-04-12T19:51:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.234+0000 7f1ab3e00640 1 -- 172.21.15.32:0/660436272 wait complete. 2024-04-12T19:51:58.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-12T19:51:58.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.483+0000 7fc4c6000640 1 Processor -- start 2024-04-12T19:51:58.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.483+0000 7fc4c6000640 1 -- start start 2024-04-12T19:51:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 0x7fc4c00b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c0162dd0 con 0x7fc4c015c680 2024-04-12T19:51:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00d34f0 con 0x7fc4c00d2b50 2024-04-12T19:51:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c0162b90 con 0x7fc4c0162730 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 0x7fc4c00b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.485+0000 7fc4c5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:33140/0 (socket says 172.21.15.32:33140) 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 0x7fc4c00b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:52590/0 (socket says 172.21.15.32:52590) 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 -- 172.21.15.32:0/1562675583 learned_addr learned my addr 172.21.15.32:0/1562675583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c4c00640 1 --1- 172.21.15.32:0/1562675583 >> v1:172.21.15.169:6789/0 conn(0x7fc4c015c680 0x7fc4c015f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:44876/0 (socket says 172.21.15.32:44876) 2024-04-12T19:51:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 -- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 msgr2=0x7fc4c00b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 --2- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 0x7fc4c00b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 -- 172.21.15.32:0/1562675583 >> v1:172.21.15.169:6789/0 conn(0x7fc4c015c680 legacy=0x7fc4c015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.486+0000 7fc4c5600640 1 -- 172.21.15.32:0/1562675583 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01630b0 con 0x7fc4c00d2b50 2024-04-12T19:51:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.488+0000 7fc4c5600640 1 --2- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95159921a59ccca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.488+0000 7fc4bf400640 1 -- 172.21.15.32:0/1562675583 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802cb00 con 0x7fc4c00d2b50 2024-04-12T19:51:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.488+0000 7fc4bf400640 1 -- 172.21.15.32:0/1562675583 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b802cca0 con 0x7fc4c00d2b50 2024-04-12T19:51:58.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.488+0000 7fc4bf400640 1 -- 172.21.15.32:0/1562675583 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8034610 con 0x7fc4c00d2b50 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 -- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c00d2b50 msgr2=0x7fc4c00d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 --2- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b802bf60 comp rx=0 tx=0).stop 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 -- 172.21.15.32:0/1562675583 shutdown_connections 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 --2- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c00d2b50 0x7fc4c00d2fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 --2- 172.21.15.32:0/1562675583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c0162730 0x7fc4c00b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 -- 172.21.15.32:0/1562675583 >> 172.21.15.32:0/1562675583 conn(0x7fc4c0073480 msgr2=0x7fc4c0073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.489+0000 7fc4c6000640 1 -- 172.21.15.32:0/1562675583 shutdown_connections 2024-04-12T19:51:58.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.490+0000 7fc4c6000640 1 -- 172.21.15.32:0/1562675583 wait complete. 2024-04-12T19:51:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.491+0000 7fc4c6000640 1 Processor -- start 2024-04-12T19:51:58.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.491+0000 7fc4c6000640 1 -- start start 2024-04-12T19:51:58.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c00d2b50 0x7fc4c0159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 0x7fc4c015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c015ca60 con 0x7fc4c015c680 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c015f330 con 0x7fc4c0162730 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.492+0000 7fc4c6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c015ff60 con 0x7fc4c00d2b50 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42064/0 (socket says 172.21.15.32:42064) 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 -- 172.21.15.32:0/2340304348 learned_addr learned my addr 172.21.15.32:0/2340304348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4c5600640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 0x7fc4c015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 -- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c00d2b50 msgr2=0x7fc4c0159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c00d2b50 0x7fc4c0159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 -- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 msgr2=0x7fc4c015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 0x7fc4c015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 -- 172.21.15.32:0/2340304348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c015a430 con 0x7fc4c015c680 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4c5600640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 0x7fc4c015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:58.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.493+0000 7fc4bfe00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e870 tx=0x7fc4ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.494+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac017070 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.494+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ac00c940 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.494+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00cd30 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.494+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c015a710 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.494+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c015acd0 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.495+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c00755c0 con 0x7fc4c015c680 2024-04-12T19:51:58.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.496+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4ac01a070 con 0x7fc4c015c680 2024-04-12T19:51:58.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.497+0000 7fc4bcc00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.497+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc4ac08bdb0 con 0x7fc4c015c680 2024-04-12T19:51:58.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.497+0000 7fc4c4c00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.498+0000 7fc4c4c00640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 0x7fc4a006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40015e0 tx=0x7fc4b40079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.503+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4ac090070 con 0x7fc4c015c680 2024-04-12T19:51:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.710+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4c006dbe0 con 0x7fc4c015c680 2024-04-12T19:51:58.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.711+0000 7fc4bcc00640 1 -- 172.21.15.32:0/2340304348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fc4ac05a1d0 con 0x7fc4c015c680 2024-04-12T19:51:58.711 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-12T19:51:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.713+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 msgr2=0x7fc4a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.713+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 0x7fc4a006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40015e0 tx=0x7fc4b40079c0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.713+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 msgr2=0x7fc4c0155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.713+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e870 tx=0x7fc4ac00ed30 comp rx=0 tx=0).stop 2024-04-12T19:51:58.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 shutdown_connections 2024-04-12T19:51:58.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4c0162730 0x7fc4c015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015c680 0x7fc4c0155030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 --2- 172.21.15.32:0/2340304348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4c00d2b50 0x7fc4c0159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 >> 172.21.15.32:0/2340304348 conn(0x7fc4c0073480 msgr2=0x7fc4c00b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 shutdown_connections 2024-04-12T19:51:58.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.714+0000 7fc4c6000640 1 -- 172.21.15.32:0/2340304348 wait complete. 2024-04-12T19:51:58.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.948+0000 7f749fe00640 1 Processor -- start 2024-04-12T19:51:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.948+0000 7f749fe00640 1 -- start start 2024-04-12T19:51:58.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a00cee00 0x7f74a006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74a015d2f0 0x7f74a00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74a015dbb0 con 0x7f74a0071f50 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749fe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f74a006f4f0 con 0x7f74a015d2f0 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f74a00728f0 con 0x7f74a00cee00 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74a015d2f0 0x7f74a00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74a015d2f0 0x7f74a00723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42072/0 (socket says 172.21.15.32:42072) 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749f400640 1 -- 172.21.15.32:0/1978364434 learned_addr learned my addr 172.21.15.32:0/1978364434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.950+0000 7f749ea00640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a00cee00 0x7f74a006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.951+0000 7f749f400640 1 -- 172.21.15.32:0/1978364434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a00cee00 msgr2=0x7f74a006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.951+0000 7f749f400640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a00cee00 0x7f74a006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.951+0000 7f749f400640 1 -- 172.21.15.32:0/1978364434 >> v1:172.21.15.169:6789/0 conn(0x7f74a0071f50 legacy=0x7f74a015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.951+0000 7f749f400640 1 -- 172.21.15.32:0/1978364434 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a015d8d0 con 0x7f74a015d2f0 2024-04-12T19:51:58.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.952+0000 7f749f400640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a015d2f0 0x7f74a00723b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e80 tx=0x7f749802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c49c9cc4f994f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.952+0000 7f749d600640 1 -- 172.21.15.32:0/1978364434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749802caf0 con 0x7f74a015d2f0 2024-04-12T19:51:58.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.952+0000 7f749d600640 1 -- 172.21.15.32:0/1978364434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749802cc90 con 0x7f74a015d2f0 2024-04-12T19:51:58.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.952+0000 7f749d600640 1 -- 172.21.15.32:0/1978364434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498034610 con 0x7f74a015d2f0 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 -- 172.21.15.32:0/1978364434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a015d2f0 msgr2=0x7f74a00723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a015d2f0 0x7f74a00723b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e80 tx=0x7f749802bf50 comp rx=0 tx=0).stop 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 -- 172.21.15.32:0/1978364434 shutdown_connections 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a015d2f0 0x7f74a00723b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 --2- 172.21.15.32:0/1978364434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a00cee00 0x7f74a006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 -- 172.21.15.32:0/1978364434 >> 172.21.15.32:0/1978364434 conn(0x7f74a006b930 msgr2=0x7f74a006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 -- 172.21.15.32:0/1978364434 shutdown_connections 2024-04-12T19:51:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.953+0000 7f749fe00640 1 -- 172.21.15.32:0/1978364434 wait complete. 2024-04-12T19:51:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.955+0000 7f749fe00640 1 Processor -- start 2024-04-12T19:51:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.955+0000 7f749fe00640 1 -- start start 2024-04-12T19:51:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a0071f50 0x7f74a014b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74a015d2f0 0x7f74a0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74a006e3a0 con 0x7f74a00cee00 2024-04-12T19:51:58.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f74a006e0c0 con 0x7f74a015d2f0 2024-04-12T19:51:58.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.956+0000 7f749fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f74a015dda0 con 0x7f74a0071f50 2024-04-12T19:51:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42088/0 (socket says 172.21.15.32:42088) 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 -- 172.21.15.32:0/2657907696 learned_addr learned my addr 172.21.15.32:0/2657907696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749f400640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74a015d2f0 0x7f74a0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749ea00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a0071f50 0x7f74a014b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 -- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a0071f50 msgr2=0x7f74a014b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a0071f50 0x7f74a014b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 -- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74a015d2f0 msgr2=0x7f74a0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74a015d2f0 0x7f74a0155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.957+0000 7f749e000640 1 -- 172.21.15.32:0/2657907696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a015e300 con 0x7f74a00cee00 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f749e000640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c430 tx=0x7f748c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0052d0 con 0x7f74a00cee00 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748c005490 con 0x7f74a00cee00 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c007260 con 0x7f74a00cee00 2024-04-12T19:51:58.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a015e5e0 con 0x7f74a00cee00 2024-04-12T19:51:58.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.958+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a015eba0 con 0x7f74a00cee00 2024-04-12T19:51:58.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.959+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a00755c0 con 0x7f74a00cee00 2024-04-12T19:51:58.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.960+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f748c007400 con 0x7f74a00cee00 2024-04-12T19:51:58.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.962+0000 7f7496a00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 0x7f748006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:58.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.962+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f748c08af70 con 0x7f74a00cee00 2024-04-12T19:51:58.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.962+0000 7f749ea00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 0x7f748006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:58.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.963+0000 7f749ea00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 0x7f748006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:58.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:58.966+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f748c058de0 con 0x7f74a00cee00 2024-04-12T19:51:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.216+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74a015c7b0 con 0x7f74a00cee00 2024-04-12T19:51:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.217+0000 7f7496a00640 1 -- 172.21.15.32:0/2657907696 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f748c05d0e0 con 0x7f74a00cee00 2024-04-12T19:51:59.218 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:59.218 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:51:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:51:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 msgr2=0x7f748006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 0x7f748006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490008040 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 msgr2=0x7f74a0155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c430 tx=0x7f748c00c8f0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 shutdown_connections 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f748006c8f0 0x7f748006edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74a015d2f0 0x7f74a0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74a00cee00 0x7f74a0155790 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 --2- 172.21.15.32:0/2657907696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74a0071f50 0x7f74a014b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 >> 172.21.15.32:0/2657907696 conn(0x7f74a006b930 msgr2=0x7f74a0154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 shutdown_connections 2024-04-12T19:51:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.220+0000 7f749fe00640 1 -- 172.21.15.32:0/2657907696 wait complete. 2024-04-12T19:51:59.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:51:59.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.465+0000 7f24b1c00640 1 Processor -- start 2024-04-12T19:51:59.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.466+0000 7f24b1c00640 1 -- start start 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.467+0000 7f24b1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f24ac162020 0x7f24ac162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.467+0000 7f24b1c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac0d0090 con 0x7f24ac0cf110 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.467+0000 7f24b1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac0cfad0 con 0x7f24ac0d1c90 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.467+0000 7f24b1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f24ac163440 con 0x7f24ac162020 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.468+0000 7f24ab400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f24ac0cf110 0x7f24ac0d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46156/0 (socket says 172.21.15.32:46156) 2024-04-12T19:51:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.468+0000 7f24ab400640 1 -- 172.21.15.32:0/485105562 learned_addr learned my addr 172.21.15.32:0/485105562 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.468+0000 7f24abe00640 1 --2- 172.21.15.32:0/485105562 >> v2:172.21.15.169:3300/0 conn(0x7f24ac162020 0x7f24ac162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.468+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309043434 0 0) 0x7f249c001490 con 0x7f24ac0cf110 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.468+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2490003520 con 0x7f24ac0cf110 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.469+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2512713885 0 0) 0x7f2490003520 con 0x7f24ac0cf110 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.469+0000 7f24abe00640 1 -- 172.21.15.32:0/485105562 >> v1:172.21.15.169:6789/0 conn(0x7f24ac0d1c90 legacy=0x7f24ac0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.469+0000 7f24abe00640 1 -- 172.21.15.32:0/485105562 >> v1:172.21.15.32:6789/0 conn(0x7f24ac0cf110 legacy=0x7f24ac0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.469+0000 7f24abe00640 1 -- 172.21.15.32:0/485105562 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac0cfdb0 con 0x7f24ac162020 2024-04-12T19:51:59.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.470+0000 7f24abe00640 1 --2- 172.21.15.32:0/485105562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac162f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24cd2f45a2be2da server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494038070 con 0x7f24ac162020 2024-04-12T19:51:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2494031500 con 0x7f24ac162020 2024-04-12T19:51:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24aaa00640 1 -- 172.21.15.32:0/485105562 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c001bc0 con 0x7f24ac162020 2024-04-12T19:51:59.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 -- 172.21.15.32:0/485105562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 msgr2=0x7f24ac162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 --2- 172.21.15.32:0/485105562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac162f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b820 comp rx=0 tx=0).stop 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 -- 172.21.15.32:0/485105562 shutdown_connections 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 --2- 172.21.15.32:0/485105562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac162f00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 -- 172.21.15.32:0/485105562 >> 172.21.15.32:0/485105562 conn(0x7f24ac06cf00 msgr2=0x7f24ac06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.471+0000 7f24b1c00640 1 -- 172.21.15.32:0/485105562 shutdown_connections 2024-04-12T19:51:59.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.472+0000 7f24b1c00640 1 -- 172.21.15.32:0/485105562 wait complete. 2024-04-12T19:51:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.473+0000 7f24b1c00640 1 Processor -- start 2024-04-12T19:51:59.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.473+0000 7f24b1c00640 1 -- start start 2024-04-12T19:51:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24ac0cf110 0x7f24ac1606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac0cf880 con 0x7f24ac0d1c90 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac0b7b30 con 0x7f24ac162020 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.474+0000 7f24b1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac0d2e70 con 0x7f24ac0cf110 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42094/0 (socket says 172.21.15.32:42094) 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 -- 172.21.15.32:0/1493759692 learned_addr learned my addr 172.21.15.32:0/1493759692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24b1200640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24abe00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24ac0cf110 0x7f24ac1606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 -- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24ac0cf110 msgr2=0x7f24ac1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24ac0cf110 0x7f24ac1606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 -- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 msgr2=0x7f24ac15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac15d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.475+0000 7f24ab400640 1 -- 172.21.15.32:0/1493759692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1f4bf0 con 0x7f24ac0d1c90 2024-04-12T19:51:59.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24ab400640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f249c00cc30 tx=0x7f249c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c017070 con 0x7f24ac0d1c90 2024-04-12T19:51:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c00ce40 con 0x7f24ac0d1c90 2024-04-12T19:51:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac15dee0 con 0x7f24ac0d1c90 2024-04-12T19:51:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c012640 con 0x7f24ac0d1c90 2024-04-12T19:51:59.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.476+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1f4e90 con 0x7f24ac0d1c90 2024-04-12T19:51:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.477+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac0755c0 con 0x7f24ac0d1c90 2024-04-12T19:51:59.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.478+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f249c01a070 con 0x7f24ac0d1c90 2024-04-12T19:51:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.479+0000 7f24a3e00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.479+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f249c08ba20 con 0x7f24ac0d1c90 2024-04-12T19:51:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.480+0000 7f24abe00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.480+0000 7f24abe00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 0x7f248806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2494035000 tx=0x7f2494008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.484+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24ac0d0090 con 0x7f24ac0d1c90 2024-04-12T19:51:59.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.688+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ac0b7650 con 0x7f24ac0d1c90 2024-04-12T19:51:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.690+0000 7f24a3e00640 1 -- 172.21.15.32:0/1493759692 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f249c059760 con 0x7f24ac0d1c90 2024-04-12T19:51:59.691 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:51:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:38.305869+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.693+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 msgr2=0x7f248806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.693+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 0x7f248806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2494035000 tx=0x7f2494008030 comp rx=0 tx=0).stop 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.693+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 msgr2=0x7f24ac15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.693+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f249c00cc30 tx=0x7f249c00a430 comp rx=0 tx=0).stop 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 shutdown_connections 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f248806c8f0 0x7f248806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24ac162020 0x7f24ac15d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24ac0d1c90 0x7f24ac15d190 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 --2- 172.21.15.32:0/1493759692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24ac0cf110 0x7f24ac1606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 >> 172.21.15.32:0/1493759692 conn(0x7f24ac06cf00 msgr2=0x7f24ac0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 shutdown_connections 2024-04-12T19:51:59.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.694+0000 7f24b1c00640 1 -- 172.21.15.32:0/1493759692 wait complete. 2024-04-12T19:51:59.714 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-12T19:51:59.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:51:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.940+0000 7f1db7200640 1 Processor -- start 2024-04-12T19:51:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.941+0000 7f1db7200640 1 -- start start 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.942+0000 7f1db7200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db00cee00 0x7f1db006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.942+0000 7f1db7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db00723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.942+0000 7f1db7200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db015dbb0 con 0x7f1db015cbe0 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.942+0000 7f1db7200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1db006f4f0 con 0x7f1db00cee00 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.942+0000 7f1db7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1db00728f0 con 0x7f1db0071f50 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db00cee00 0x7f1db006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db6800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1db015cbe0 0x7f1db015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46172/0 (socket says 172.21.15.32:46172) 2024-04-12T19:51:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db6800640 1 -- 172.21.15.32:0/3252673294 learned_addr learned my addr 172.21.15.32:0/3252673294 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5400640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db00723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db4a00640 1 -- 172.21.15.32:0/3252673294 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492575814 0 0) 0x7f1db015dbb0 con 0x7f1db015cbe0 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db4a00640 1 -- 172.21.15.32:0/3252673294 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d9c003520 con 0x7f1db015cbe0 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5e00640 1 -- 172.21.15.32:0/3252673294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 msgr2=0x7f1db00723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5e00640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db00723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5e00640 1 -- 172.21.15.32:0/3252673294 >> v1:172.21.15.32:6789/0 conn(0x7f1db015cbe0 legacy=0x7f1db015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.943+0000 7f1db5e00640 1 -- 172.21.15.32:0/3252673294 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db015d8d0 con 0x7f1db00cee00 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db5e00640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db00cee00 0x7f1db006e8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009dc0 tx=0x7f1da802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a925ded945fe306b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db4a00640 1 -- 172.21.15.32:0/3252673294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8039070 con 0x7f1db00cee00 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db4a00640 1 -- 172.21.15.32:0/3252673294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da802cc40 con 0x7f1db00cee00 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db4a00640 1 -- 172.21.15.32:0/3252673294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8034440 con 0x7f1db00cee00 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db7200640 1 -- 172.21.15.32:0/3252673294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db00cee00 msgr2=0x7f1db006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.945+0000 7f1db7200640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db00cee00 0x7f1db006e8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009dc0 tx=0x7f1da802be90 comp rx=0 tx=0).stop 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 -- 172.21.15.32:0/3252673294 shutdown_connections 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db00723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 --2- 172.21.15.32:0/3252673294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db00cee00 0x7f1db006e8c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 -- 172.21.15.32:0/3252673294 >> 172.21.15.32:0/3252673294 conn(0x7f1db006b930 msgr2=0x7f1db006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:51:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 -- 172.21.15.32:0/3252673294 shutdown_connections 2024-04-12T19:51:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.946+0000 7f1db7200640 1 -- 172.21.15.32:0/3252673294 wait complete. 2024-04-12T19:51:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.947+0000 7f1db7200640 1 Processor -- start 2024-04-12T19:51:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.948+0000 7f1db7200640 1 -- start start 2024-04-12T19:51:59.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db0148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 0x7f1db00aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1db006e3a0 con 0x7f1db015cbe0 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1db006e0c0 con 0x7f1db00cee00 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1db015dda0 con 0x7f1db0071f50 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42110/0 (socket says 172.21.15.32:42110) 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 -- 172.21.15.32:0/1614900874 learned_addr learned my addr 172.21.15.32:0/1614900874 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db5400640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 0x7f1db00aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db5e00640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db0148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 -- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 msgr2=0x7f1db0148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db0148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 -- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 msgr2=0x7f1db00aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:51:59.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 0x7f1db00aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:51:59.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db6800640 1 -- 172.21.15.32:0/1614900874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db00b5070 con 0x7f1db015cbe0 2024-04-12T19:51:59.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.949+0000 7f1db5400640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 0x7f1db00aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:51:59.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1db6800640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c390 tx=0x7f1dac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac017070 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dac005ce0 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac007310 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db00b5350 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.950+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db00ae2c0 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.951+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db00755c0 con 0x7f1db015cbe0 2024-04-12T19:51:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.953+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dac01a070 con 0x7f1db015cbe0 2024-04-12T19:51:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.954+0000 7f1da6000640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 0x7f1d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:51:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.954+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1dac08aff0 con 0x7f1db015cbe0 2024-04-12T19:51:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.954+0000 7f1db5e00640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 0x7f1d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:51:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.955+0000 7f1db5e00640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 0x7f1d9406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1da8036000 tx=0x7f1da8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:51:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:51:59.959+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1dac0593a0 con 0x7f1db015cbe0 2024-04-12T19:52:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.205+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1db015d350 con 0x7f1db015cbe0 2024-04-12T19:52:00.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.206+0000 7f1da6000640 1 -- 172.21.15.32:0/1614900874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f1dac058ca0 con 0x7f1db015cbe0 2024-04-12T19:52:00.207 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:00.207 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:52:00.207 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:52:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 msgr2=0x7f1d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 0x7f1d9406edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1da8036000 tx=0x7f1da8030000 comp rx=0 tx=0).stop 2024-04-12T19:52:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 msgr2=0x7f1db0149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00c390 tx=0x7f1dac00c850 comp rx=0 tx=0).stop 2024-04-12T19:52:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 shutdown_connections 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1d9406c8f0 0x7f1d9406edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db015cbe0 0x7f1db0149870 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db00cee00 0x7f1db00aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 --2- 172.21.15.32:0/1614900874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1db0071f50 0x7f1db0148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 >> 172.21.15.32:0/1614900874 conn(0x7f1db006b930 msgr2=0x7f1db006e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 shutdown_connections 2024-04-12T19:52:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.210+0000 7f1db7200640 1 -- 172.21.15.32:0/1614900874 wait complete. 2024-04-12T19:52:00.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:52:00.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.458+0000 7f82ed600640 1 Processor -- start 2024-04-12T19:52:00.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.459+0000 7f82ed600640 1 -- start start 2024-04-12T19:52:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.460+0000 7f82ed600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82e815ea70 0x7f82e80b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.460+0000 7f82ed600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82e80b0280 0x7f82e80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.460+0000 7f82ed600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e80b1860 con 0x7f82e80b12e0 2024-04-12T19:52:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.460+0000 7f82ed600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f82e80b1b40 con 0x7f82e80b0280 2024-04-12T19:52:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.460+0000 7f82ed600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f82e80b1e20 con 0x7f82e815ea70 2024-04-12T19:52:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82e80b0280 0x7f82e80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e6000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f82e80b12e0 0x7f82e806b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46174/0 (socket says 172.21.15.32:46174) 2024-04-12T19:52:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e6000640 1 -- 172.21.15.32:0/1620935360 learned_addr learned my addr 172.21.15.32:0/1620935360 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:00.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e6a00640 1 --2- 172.21.15.32:0/1620935360 >> v2:172.21.15.169:3300/0 conn(0x7f82e815ea70 0x7f82e80b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e5600640 1 -- 172.21.15.32:0/1620935360 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525630068 0 0) 0x7f82e80b1860 con 0x7f82e80b12e0 2024-04-12T19:52:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.461+0000 7f82e5600640 1 -- 172.21.15.32:0/1620935360 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82cc003520 con 0x7f82e80b12e0 2024-04-12T19:52:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.462+0000 7f82e7400640 1 -- 172.21.15.32:0/1620935360 >> v2:172.21.15.169:3300/0 conn(0x7f82e815ea70 msgr2=0x7f82e80b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.462+0000 7f82e7400640 1 --2- 172.21.15.32:0/1620935360 >> v2:172.21.15.169:3300/0 conn(0x7f82e815ea70 0x7f82e80b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.462+0000 7f82e7400640 1 -- 172.21.15.32:0/1620935360 >> v1:172.21.15.32:6789/0 conn(0x7f82e80b12e0 legacy=0x7f82e806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.462+0000 7f82e7400640 1 -- 172.21.15.32:0/1620935360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e80b2100 con 0x7f82e80b0280 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.463+0000 7f82e7400640 1 --2- 172.21.15.32:0/1620935360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b0280 0x7f82e80b5860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82e8073610 tx=0x7f82e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6e90b9e0066b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.463+0000 7f82e5600640 1 -- 172.21.15.32:0/1620935360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0035020 con 0x7f82e80b0280 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.463+0000 7f82e5600640 1 -- 172.21.15.32:0/1620935360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e0031830 con 0x7f82e80b0280 2024-04-12T19:52:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.463+0000 7f82e5600640 1 -- 172.21.15.32:0/1620935360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0031c00 con 0x7f82e80b0280 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 -- 172.21.15.32:0/1620935360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b0280 msgr2=0x7f82e80b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 --2- 172.21.15.32:0/1620935360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b0280 0x7f82e80b5860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82e8073610 tx=0x7f82e002b8d0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 -- 172.21.15.32:0/1620935360 shutdown_connections 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 --2- 172.21.15.32:0/1620935360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b0280 0x7f82e80b5860 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 --2- 172.21.15.32:0/1620935360 >> v2:172.21.15.169:3300/0 conn(0x7f82e815ea70 0x7f82e80b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 -- 172.21.15.32:0/1620935360 >> 172.21.15.32:0/1620935360 conn(0x7f82e80b6f50 msgr2=0x7f82e809aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 -- 172.21.15.32:0/1620935360 shutdown_connections 2024-04-12T19:52:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.464+0000 7f82ed600640 1 -- 172.21.15.32:0/1620935360 wait complete. 2024-04-12T19:52:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.466+0000 7f82ed600640 1 Processor -- start 2024-04-12T19:52:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.466+0000 7f82ed600640 1 -- start start 2024-04-12T19:52:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e80b0280 0x7f82e8155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e815ea70 0x7f82e806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815efd0 con 0x7f82e80b12e0 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815ee50 con 0x7f82e80b0280 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82ed600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82e809b2a0 con 0x7f82e815ea70 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42118/0 (socket says 172.21.15.32:42118) 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 -- 172.21.15.32:0/1467388912 learned_addr learned my addr 172.21.15.32:0/1467388912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 -- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e815ea70 msgr2=0x7f82e806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e815ea70 0x7f82e806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 -- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e80b0280 msgr2=0x7f82e8155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e80b0280 0x7f82e8155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.467+0000 7f82e6000640 1 -- 172.21.15.32:0/1467388912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e8154500 con 0x7f82e80b12e0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82e6000640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82d800cbb0 tx=0x7f82d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8017070 con 0x7f82e80b12e0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d800ad60 con 0x7f82e80b12e0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8012640 con 0x7f82e80b12e0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e8151800 con 0x7f82e80b12e0 2024-04-12T19:52:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.468+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e8154780 con 0x7f82e80b12e0 2024-04-12T19:52:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.469+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e80aef50 con 0x7f82e80b12e0 2024-04-12T19:52:00.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.471+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82d801a070 con 0x7f82e80b12e0 2024-04-12T19:52:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.472+0000 7f82dea00640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.472+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f82d808b9b0 con 0x7f82e80b12e0 2024-04-12T19:52:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.472+0000 7f82e6a00640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.473+0000 7f82e6a00640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 0x7f82c406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82d0006fd0 tx=0x7f82d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.476+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82e80aef50 con 0x7f82e80b12e0 2024-04-12T19:52:00.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.722+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82e80b4ae0 con 0x7f82e80b12e0 2024-04-12T19:52:00.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.723+0000 7f82dea00640 1 -- 172.21.15.32:0/1467388912 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f82e80b4ae0 con 0x7f82e80b12e0 2024-04-12T19:52:00.723 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:00.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T19:51:36.786433+0000","modified":"2024-04-12T19:51:39.843794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/1336962272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1336962272},{"type":"v1","addr":"172.21.15.32:6836","nonce":1336962272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-04-12T19:52:00.724 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2024-04-12T19:52:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 msgr2=0x7f82c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 0x7f82c406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82d0006fd0 tx=0x7f82d0008040 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 msgr2=0x7f82e81512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82d800cbb0 tx=0x7f82d800a430 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 shutdown_connections 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f82c406c8f0 0x7f82c406edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e815ea70 0x7f82e806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82e80b12e0 0x7f82e81512c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 --2- 172.21.15.32:0/1467388912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e80b0280 0x7f82e8155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 >> 172.21.15.32:0/1467388912 conn(0x7f82e80b6f50 msgr2=0x7f82e815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 shutdown_connections 2024-04-12T19:52:00.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.726+0000 7f82ed600640 1 -- 172.21.15.32:0/1467388912 wait complete. 2024-04-12T19:52:00.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:52:00.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.969+0000 7f07d5e00640 1 Processor -- start 2024-04-12T19:52:00.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.969+0000 7f07d5e00640 1 -- start start 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.970+0000 7f07d5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07d015c7e0 0x7f07d00c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d5e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d009b780 con 0x7f07d00b3dd0 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d5e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d009ba60 con 0x7f07d00c83b0 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d5e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f07d00d0b80 con 0x7f07d015c7e0 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d5400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07d00b3dd0 0x7f07d00d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46190/0 (socket says 172.21.15.32:46190) 2024-04-12T19:52:00.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d5400640 1 -- 172.21.15.32:0/360444390 learned_addr learned my addr 172.21.15.32:0/360444390 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07d4a00640 1 --2- 172.21.15.32:0/360444390 >> v2:172.21.15.169:3300/0 conn(0x7f07d015c7e0 0x7f07d00c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3619028101 0 0) 0x7f07d009b780 con 0x7f07d00b3dd0 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.971+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07b4003520 con 0x7f07d00b3dd0 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3222422177 0 0) 0x7f07b4003520 con 0x7f07d00b3dd0 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 >> v2:172.21.15.169:3300/0 conn(0x7f07d015c7e0 msgr2=0x7f07d00c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 --2- 172.21.15.32:0/360444390 >> v2:172.21.15.169:3300/0 conn(0x7f07d015c7e0 0x7f07d00c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 >> v1:172.21.15.169:6789/0 conn(0x7f07d00c83b0 legacy=0x7f07d00b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d009bd40 con 0x7f07d00b3dd0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f07c8001670 con 0x7f07d00b3dd0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 >> v1:172.21.15.32:6789/0 conn(0x7f07d00b3dd0 legacy=0x7f07d00d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07b4003a00 0x7f07b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b40051c0 0x7f07b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07d00d0b80 con 0x7f07b4004800 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07b400c0a0 con 0x7f07b40051c0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07b400c2a0 con 0x7f07b4003a00 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.972+0000 7f07cfe00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07d4a00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07b4003a00 0x7f07b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07d5400640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b40051c0 0x7f07b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07cfe00640 1 -- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07b4003a00 msgr2=0x7f07b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:00.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07cfe00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07b4003a00 0x7f07b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07cfe00640 1 -- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b40051c0 msgr2=0x7f07b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07cfe00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b40051c0 0x7f07b400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07cfe00640 1 -- 172.21.15.32:0/360444390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b4003520 con 0x7f07b4004800 2024-04-12T19:52:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.973+0000 7f07d5400640 1 -- 172.21.15.32:0/360444390 reap_dead start 2024-04-12T19:52:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.974+0000 7f07cfe00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07b4004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07b8001750 tx=0x7f07b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b20df53a9c8e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.974+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8001370 con 0x7f07b4004800 2024-04-12T19:52:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.974+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b80052d0 con 0x7f07b4004800 2024-04-12T19:52:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.974+0000 7f07cf400640 1 -- 172.21.15.32:0/360444390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003830 con 0x7f07b4004800 2024-04-12T19:52:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 -- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 msgr2=0x7f07b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07b4004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07b8001750 tx=0x7f07b4005830 comp rx=0 tx=0).stop 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 -- 172.21.15.32:0/360444390 shutdown_connections 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 --2- 172.21.15.32:0/360444390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07b4004c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 -- 172.21.15.32:0/360444390 >> 172.21.15.32:0/360444390 conn(0x7f07d00d25e0 msgr2=0x7f07d00d29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 -- 172.21.15.32:0/360444390 shutdown_connections 2024-04-12T19:52:00.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.975+0000 7f07d5e00640 1 -- 172.21.15.32:0/360444390 wait complete. 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.977+0000 7f07d5e00640 1 Processor -- start 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.977+0000 7f07d5e00640 1 -- start start 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07d0154f80 0x7f07d015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07d015dc90 0x7f07d00cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07d015cbc0 con 0x7f07b4004800 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07d009b450 con 0x7f07d015dc90 2024-04-12T19:52:00.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07d00c7e70 con 0x7f07d0154f80 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42136/0 (socket says 172.21.15.32:42136) 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 -- 172.21.15.32:0/4044332317 learned_addr learned my addr 172.21.15.32:0/4044332317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 -- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07d0154f80 msgr2=0x7f07d015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07d0154f80 0x7f07d015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d5400640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07d015dc90 0x7f07d00cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.978+0000 7f07d4a00640 1 -- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07d015dc90 msgr2=0x7f07d00cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07d4a00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07d015dc90 0x7f07d00cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07d4a00640 1 -- 172.21.15.32:0/4044332317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d0155400 con 0x7f07b4004800 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07d4a00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007f30 tx=0x7f07c0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001a020 con 0x7f07b4004800 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c0016470 con 0x7f07b4004800 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0016860 con 0x7f07b4004800 2024-04-12T19:52:00.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d015e720 con 0x7f07b4004800 2024-04-12T19:52:00.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.979+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d015ece0 con 0x7f07b4004800 2024-04-12T19:52:00.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.981+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d00d3240 con 0x7f07b4004800 2024-04-12T19:52:00.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.982+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07c0005d10 con 0x7f07b4004800 2024-04-12T19:52:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.982+0000 7f07ccc00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 0x7f07ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.983+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f07c008ca90 con 0x7f07b4004800 2024-04-12T19:52:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.983+0000 7f07cfe00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 0x7f07ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:00.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.984+0000 7f07cfe00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 0x7f07ac06ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07b8033000 tx=0x7f07b802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:00.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:00.987+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07c0024030 con 0x7f07b4004800 2024-04-12T19:52:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.193+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07d0070f80 con 0x7f07b4004800 2024-04-12T19:52:01.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.196+0000 7f07ccc00640 1 -- 172.21.15.32:0/4044332317 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f07c005aec0 con 0x7f07b4004800 2024-04-12T19:52:01.197 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:01.197 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:51:38.305869+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-12T19:51:34.775305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-12T19:51:35.764856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:52:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.198+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 msgr2=0x7f07ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.198+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 0x7f07ac06ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07b8033000 tx=0x7f07b802f000 comp rx=0 tx=0).stop 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.198+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 msgr2=0x7f07d015f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.198+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007f30 tx=0x7f07c0011660 comp rx=0 tx=0).stop 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 shutdown_connections 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f07ac06c920 0x7f07ac06ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07d015dc90 0x7f07d00cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07d0154f80 0x7f07d015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 --2- 172.21.15.32:0/4044332317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4004800 0x7f07d015f7c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 >> 172.21.15.32:0/4044332317 conn(0x7f07d00d25e0 msgr2=0x7f07d0071260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 shutdown_connections 2024-04-12T19:52:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.199+0000 7f07d5e00640 1 -- 172.21.15.32:0/4044332317 wait complete. 2024-04-12T19:52:01.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-12T19:52:01.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.452+0000 7f5dc9c00640 1 Processor -- start 2024-04-12T19:52:01.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.452+0000 7f5dc9c00640 1 -- start start 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5dc40afff0 0x7f5dc40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc9c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40cfd50 con 0x7f5dc4144e40 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc9c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40d0030 con 0x7f5dc406f210 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc40b3f30 con 0x7f5dc40afff0 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc2000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5dc4144e40 0x7f5dc415bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46206/0 (socket says 172.21.15.32:46206) 2024-04-12T19:52:01.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.454+0000 7f5dc2000640 1 -- 172.21.15.32:0/3627579267 learned_addr learned my addr 172.21.15.32:0/3627579267 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:01.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234421819 0 0) 0x7f5dc40cfd50 con 0x7f5dc4144e40 2024-04-12T19:52:01.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc2a00640 1 --2- 172.21.15.32:0/3627579267 >> v2:172.21.15.169:3300/0 conn(0x7f5dc40afff0 0x7f5dc40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5da8003520 con 0x7f5dc4144e40 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474090343 0 0) 0x7f5da8003520 con 0x7f5dc4144e40 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 >> v2:172.21.15.169:3300/0 conn(0x7f5dc40afff0 msgr2=0x7f5dc40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 --2- 172.21.15.32:0/3627579267 >> v2:172.21.15.169:3300/0 conn(0x7f5dc40afff0 0x7f5dc40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 >> v1:172.21.15.169:6789/0 conn(0x7f5dc406f210 legacy=0x7f5dc40b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40d0310 con 0x7f5dc4144e40 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f5db0001670 con 0x7f5dc4144e40 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 >> v1:172.21.15.32:6789/0 conn(0x7f5dc4144e40 legacy=0x7f5dc415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8003a00 0x7f5da8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 0x7f5da8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5da8005290 0x7f5da800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40b3f30 con 0x7f5da8004800 2024-04-12T19:52:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5da800c1e0 con 0x7f5da8003a00 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5da800c3e0 con 0x7f5da8005290 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.455+0000 7f5dc2000640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 0x7f5da8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2a00640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8003a00 0x7f5da8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2000640 1 -- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5da8005290 msgr2=0x7f5da800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc3400640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5da8005290 0x7f5da800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2000640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5da8005290 0x7f5da800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2000640 1 -- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8003a00 msgr2=0x7f5da8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2000640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8003a00 0x7f5da8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc2000640 1 -- 172.21.15.32:0/3627579267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8003520 con 0x7f5da8004800 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc3400640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5da8005290 0x7f5da800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.456+0000 7f5dc3400640 1 -- 172.21.15.32:0/3627579267 reap_dead start 2024-04-12T19:52:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.457+0000 7f5dc2000640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 0x7f5da8004c80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c110 tx=0x7f5da8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93805aff51ad6f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0007c40 con 0x7f5da8004800 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db0030070 con 0x7f5da8004800 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc1600640 1 -- 172.21.15.32:0/3627579267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0005d70 con 0x7f5da8004800 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 msgr2=0x7f5da8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 0x7f5da8004c80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c110 tx=0x7f5da8005830 comp rx=0 tx=0).stop 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3627579267 shutdown_connections 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3627579267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5da8004800 0x7f5da8004c80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3627579267 >> 172.21.15.32:0/3627579267 conn(0x7f5dc40b2460 msgr2=0x7f5dc40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3627579267 shutdown_connections 2024-04-12T19:52:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.458+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3627579267 wait complete. 2024-04-12T19:52:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.460+0000 7f5dc9c00640 1 Processor -- start 2024-04-12T19:52:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.460+0000 7f5dc9c00640 1 -- start start 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8004800 0x7f5dc40cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc415cd70 0x7f5dc415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40cf690 con 0x7f5dc40adba0 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40b4210 con 0x7f5da8004800 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.461+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40cf810 con 0x7f5dc415cd70 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42150/0 (socket says 172.21.15.32:42150) 2024-04-12T19:52:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 -- 172.21.15.32:0/3678236172 learned_addr learned my addr 172.21.15.32:0/3678236172 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc2a00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8004800 0x7f5dc40cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc2000640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc415cd70 0x7f5dc415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 -- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc415cd70 msgr2=0x7f5dc415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc415cd70 0x7f5dc415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 -- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8004800 msgr2=0x7f5dc40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8004800 0x7f5dc40cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.462+0000 7f5dc3400640 1 -- 172.21.15.32:0/3678236172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40cc8d0 con 0x7f5dc40adba0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dc3400640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5da80057e0 tx=0x7f5dbc00d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc005440 con 0x7f5dc40adba0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc007020 con 0x7f5dc40adba0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc007410 con 0x7f5dc40adba0 2024-04-12T19:52:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc415cae0 con 0x7f5dc40adba0 2024-04-12T19:52:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.463+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc415d5c0 con 0x7f5dc40adba0 2024-04-12T19:52:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.464+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4068c80 con 0x7f5dc40adba0 2024-04-12T19:52:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.466+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dbc0067d0 con 0x7f5dc40adba0 2024-04-12T19:52:01.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.466+0000 7f5dbaa00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.467+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5dbc08b090 con 0x7f5dc40adba0 2024-04-12T19:52:01.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.467+0000 7f5dc2a00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.467+0000 7f5dc2a00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db4003fd0 tx=0x7f5db400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.471+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dc40cc8d0 con 0x7f5dc40adba0 2024-04-12T19:52:01.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.715+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5dc406faa0 con 0x7f5dc40adba0 2024-04-12T19:52:01.716 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-12T19:52:01.716+0000 7f5896200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T19:52:01.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.720+0000 7f5dbaa00640 1 -- 172.21.15.32:0/3678236172 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f5dbc0594c0 con 0x7f5dc40adba0 2024-04-12T19:52:01.720 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-12T19:52:01.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 msgr2=0x7f5da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db4003fd0 tx=0x7f5db400b040 comp rx=0 tx=0).stop 2024-04-12T19:52:01.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 msgr2=0x7f5dc40ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5da80057e0 tx=0x7f5dbc00d7d0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 shutdown_connections 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40adba0 0x7f5dc40ae020 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc415cd70 0x7f5dc415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 --2- 172.21.15.32:0/3678236172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5da8004800 0x7f5dc40cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.723+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 >> 172.21.15.32:0/3678236172 conn(0x7f5dc40b2460 msgr2=0x7f5dc4143f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:01.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.724+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 shutdown_connections 2024-04-12T19:52:01.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.725+0000 7f5dc9c00640 1 -- 172.21.15.32:0/3678236172 wait complete. 2024-04-12T19:52:01.734 INFO:tasks.ceph.mds.c.smithi032.stderr: -4606> 2024-04-12T19:51:40.103+0000 7f1aac000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-12T19:52:01.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-12T19:52:01.804 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-12T19:52:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.942+0000 7f32c7200640 1 Processor -- start 2024-04-12T19:52:01.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.942+0000 7f32c7200640 1 -- start start 2024-04-12T19:52:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32c81478b0 0x7f32c80afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32c806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c7200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c80aebd0 con 0x7f32c806f750 2024-04-12T19:52:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f32c806fbb0 con 0x7f32c81478b0 2024-04-12T19:52:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32c80b0130 con 0x7f32c8147c90 2024-04-12T19:52:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c6800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32c806f750 0x7f32c81626c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46224/0 (socket says 172.21.15.32:46224) 2024-04-12T19:52:01.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.944+0000 7f32c6800640 1 -- 172.21.15.32:0/411514789 learned_addr learned my addr 172.21.15.32:0/411514789 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c5e00640 1 --2- 172.21.15.32:0/411514789 >> v2:172.21.15.169:3300/0 conn(0x7f32c81478b0 0x7f32c80afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c5400640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32c806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988784116 0 0) 0x7f32c0001490 con 0x7f32c806f750 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32ac003520 con 0x7f32c806f750 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4193158506 0 0) 0x7f32ac003520 con 0x7f32c806f750 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 msgr2=0x7f32c806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32c806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 >> v2:172.21.15.169:3300/0 conn(0x7f32c81478b0 msgr2=0x7f32c80afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 --2- 172.21.15.32:0/411514789 >> v2:172.21.15.169:3300/0 conn(0x7f32c81478b0 0x7f32c80afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c80aee50 con 0x7f32c806f750 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f32c0001bc0 con 0x7f32c806f750 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 >> v1:172.21.15.32:6789/0 conn(0x7f32c806f750 legacy=0x7f32c81626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac0039b0 0x7f32ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.945+0000 7f32c4a00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ac004800 0x7f32ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c4a00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80b0130 con 0x7f32ac004800 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac00b9c0 con 0x7f32ac0039b0 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac00bb40 con 0x7f32c8147c90 2024-04-12T19:52:01.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ac004800 0x7f32ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5e00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac0039b0 0x7f32ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 -- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 msgr2=0x7f32ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c6800640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8147c90 0x7f32ac0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 -- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac0039b0 msgr2=0x7f32ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac0039b0 0x7f32ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5400640 1 -- 172.21.15.32:0/411514789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac003520 con 0x7f32ac004800 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.946+0000 7f32c5e00640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac0039b0 0x7f32ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.947+0000 7f32c5400640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ac004800 0x7f32ac004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009ac0 tx=0x7f32ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a1bf35599a6254 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4039070 con 0x7f32ac004800 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b402c6b0 con 0x7f32ac004800 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c4a00640 1 -- 172.21.15.32:0/411514789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4009690 con 0x7f32ac004800 2024-04-12T19:52:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c7200640 1 -- 172.21.15.32:0/411514789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ac004800 msgr2=0x7f32ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c7200640 1 --2- 172.21.15.32:0/411514789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32ac004800 0x7f32ac004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009ac0 tx=0x7f32ac005830 comp rx=0 tx=0).stop 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c6800640 1 -- 172.21.15.32:0/411514789 reap_dead start 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c7200640 1 -- 172.21.15.32:0/411514789 shutdown_connections 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c7200640 1 -- 172.21.15.32:0/411514789 >> 172.21.15.32:0/411514789 conn(0x7f32c806b930 msgr2=0x7f32c806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.948+0000 7f32c7200640 1 -- 172.21.15.32:0/411514789 shutdown_connections 2024-04-12T19:52:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.949+0000 7f32c7200640 1 -- 172.21.15.32:0/411514789 wait complete. 2024-04-12T19:52:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.950+0000 7f32c7200640 1 Processor -- start 2024-04-12T19:52:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.950+0000 7f32c7200640 1 -- start start 2024-04-12T19:52:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.951+0000 7f32c7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac004800 0x7f32c8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.951+0000 7f32c7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.951+0000 7f32c7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8150d80 0x7f32c81534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.951+0000 7f32c7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80af6d0 con 0x7f32c8157530 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80af3f0 con 0x7f32ac004800 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32c806f210 con 0x7f32c8150d80 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42198/0 (socket says 172.21.15.32:42198) 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 -- 172.21.15.32:0/2622079700 learned_addr learned my addr 172.21.15.32:0/2622079700 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 -- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8150d80 msgr2=0x7f32c81534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8150d80 0x7f32c81534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5e00640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac004800 0x7f32c8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 -- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac004800 msgr2=0x7f32c8151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac004800 0x7f32c8151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.952+0000 7f32c5400640 1 -- 172.21.15.32:0/2622079700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c81579b0 con 0x7f32c8157530 2024-04-12T19:52:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32c5400640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32b4000c00 tx=0x7f32b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4042020 con 0x7f32c8157530 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b403a040 con 0x7f32c8157530 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4039070 con 0x7f32c8157530 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c81466a0 con 0x7f32c8157530 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.953+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c80cdea0 con 0x7f32c8157530 2024-04-12T19:52:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.954+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c81511a0 con 0x7f32c8157530 2024-04-12T19:52:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.955+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32b4045020 con 0x7f32c8157530 2024-04-12T19:52:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.956+0000 7f32be000640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 0x7f32a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.956+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f32b40acfe0 con 0x7f32c8157530 2024-04-12T19:52:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.957+0000 7f32c5e00640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 0x7f32a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.957+0000 7f32c5e00640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 0x7f32a806ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009a40 tx=0x7f32b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:01.961+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32b404c070 con 0x7f32c8157530 2024-04-12T19:52:02.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.208+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f32c80755c0 con 0x7f32c8157530 2024-04-12T19:52:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.785+0000 7f32be000640 1 -- 172.21.15.32:0/2622079700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f32b407b3a0 con 0x7f32c8157530 2024-04-12T19:52:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 msgr2=0x7f32a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 0x7f32a806ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009a40 tx=0x7f32b8008040 comp rx=0 tx=0).stop 2024-04-12T19:52:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 msgr2=0x7f32c8150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32b4000c00 tx=0x7f32b4004730 comp rx=0 tx=0).stop 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 shutdown_connections 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f32a806c920 0x7f32a806ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32c8150d80 0x7f32c81534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c8157530 0x7f32c8150840 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 --2- 172.21.15.32:0/2622079700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32ac004800 0x7f32c8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.789+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 >> 172.21.15.32:0/2622079700 conn(0x7f32c806b930 msgr2=0x7f32c80b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.790+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 shutdown_connections 2024-04-12T19:52:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:02.790+0000 7f32c7200640 1 -- 172.21.15.32:0/2622079700 wait complete. 2024-04-12T19:52:02.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-12T19:52:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.041+0000 7f6b25a00640 1 Processor -- start 2024-04-12T19:52:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.041+0000 7f6b25a00640 1 -- start start 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b200b52f0 0x7f6b2006f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b200b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b2015f340 con 0x7f6b2015edd0 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2015f620 con 0x7f6b200b52f0 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2015f900 con 0x7f6b200b56d0 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b2015edd0 0x7f6b20071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46234/0 (socket says 172.21.15.32:46234) 2024-04-12T19:52:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b25000640 1 -- 172.21.15.32:0/3802990920 learned_addr learned my addr 172.21.15.32:0/3802990920 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:03.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b1f400640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b200b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.044+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3802990920 >> v2:172.21.15.169:3300/0 conn(0x7f6b200b52f0 0x7f6b2006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245950270 0 0) 0x7f6b2015f340 con 0x7f6b2015edd0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b04003520 con 0x7f6b2015edd0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3521717345 0 0) 0x7f6b04003520 con 0x7f6b2015edd0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 msgr2=0x7f6b200b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b200b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 >> v2:172.21.15.169:3300/0 conn(0x7f6b200b52f0 msgr2=0x7f6b2006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 --2- 172.21.15.32:0/3802990920 >> v2:172.21.15.169:3300/0 conn(0x7f6b200b52f0 0x7f6b2006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2015fbe0 con 0x7f6b2015edd0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f6b180019f0 con 0x7f6b2015edd0 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 >> v1:172.21.15.32:6789/0 conn(0x7f6b2015edd0 legacy=0x7f6b20071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b040039b0 0x7f6b04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b04004d40 0x7f6b040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2015f620 con 0x7f6b040039b0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2015f900 con 0x7f6b04004d40 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.045+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0400b9c0 con 0x7f6b200b56d0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b040039b0 0x7f6b04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b25000640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b04004d40 0x7f6b040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1f400640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 -- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 msgr2=0x7f6b04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b200b56d0 0x7f6b04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 -- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b04004d40 msgr2=0x7f6b040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b04004d40 0x7f6b040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b1fe00640 1 -- 172.21.15.32:0/3802990920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b04003520 con 0x7f6b040039b0 2024-04-12T19:52:03.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.046+0000 7f6b25000640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b04004d40 0x7f6b040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.048+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b040039b0 0x7f6b04003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009ac0 tx=0x7f6b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4438506a8b2f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.048+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10039070 con 0x7f6b040039b0 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.048+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1002c710 con 0x7f6b040039b0 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.048+0000 7f6b1ea00640 1 -- 172.21.15.32:0/3802990920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10009690 con 0x7f6b040039b0 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 -- 172.21.15.32:0/3802990920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b040039b0 msgr2=0x7f6b04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3802990920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b040039b0 0x7f6b04003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009ac0 tx=0x7f6b04005830 comp rx=0 tx=0).stop 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25000640 1 -- 172.21.15.32:0/3802990920 reap_dead start 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 -- 172.21.15.32:0/3802990920 shutdown_connections 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 -- 172.21.15.32:0/3802990920 >> 172.21.15.32:0/3802990920 conn(0x7f6b200b6170 msgr2=0x7f6b2006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 -- 172.21.15.32:0/3802990920 shutdown_connections 2024-04-12T19:52:03.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.049+0000 7f6b25a00640 1 -- 172.21.15.32:0/3802990920 wait complete. 2024-04-12T19:52:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.051+0000 7f6b25a00640 1 Processor -- start 2024-04-12T19:52:03.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.051+0000 7f6b25a00640 1 -- start start 2024-04-12T19:52:03.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b040039b0 0x7f6b200b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b20158980 0x7f6b20142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b200727a0 con 0x7f6b20141a50 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20072620 con 0x7f6b20158980 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.052+0000 7f6b25a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b200b3650 con 0x7f6b040039b0 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42222/0 (socket says 172.21.15.32:42222) 2024-04-12T19:52:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 -- 172.21.15.32:0/3924930203 learned_addr learned my addr 172.21.15.32:0/3924930203 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b040039b0 0x7f6b200b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b25000640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b20158980 0x7f6b20142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 -- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b040039b0 msgr2=0x7f6b200b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b040039b0 0x7f6b200b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 -- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b20158980 msgr2=0x7f6b20142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b20158980 0x7f6b20142070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 -- 172.21.15.32:0/3924930203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b200b1db0 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.053+0000 7f6b1f400640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c011600 tx=0x7f6b0c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.054+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00f920 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.054+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00fac0 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.054+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00fe70 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.054+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b201f5060 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.054+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b200b2310 con 0x7f6b20141a50 2024-04-12T19:52:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.055+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b20068b40 con 0x7f6b20141a50 2024-04-12T19:52:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.056+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b0c015460 con 0x7f6b20141a50 2024-04-12T19:52:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.057+0000 7f6b17e00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 0x7f6b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.057+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f6b0c08c160 con 0x7f6b20141a50 2024-04-12T19:52:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.057+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 0x7f6b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:03.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.058+0000 7f6b1fe00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 0x7f6b0006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b10036000 tx=0x7f6b10030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:03.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.064+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b200b1db0 con 0x7f6b20141a50 2024-04-12T19:52:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.268+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b20158d60 con 0x7f6b20141a50 2024-04-12T19:52:03.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.798+0000 7f6b17e00640 1 -- 172.21.15.32:0/3924930203 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f6b0c019030 con 0x7f6b20141a50 2024-04-12T19:52:03.799 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-12T19:52:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 msgr2=0x7f6b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 0x7f6b0006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b10036000 tx=0x7f6b10030000 comp rx=0 tx=0).stop 2024-04-12T19:52:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 msgr2=0x7f6b20158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:03.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c011600 tx=0x7f6b0c011ac0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 shutdown_connections 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f6b0006c8f0 0x7f6b0006edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b20158980 0x7f6b20142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b20141a50 0x7f6b20158440 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 --2- 172.21.15.32:0/3924930203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b040039b0 0x7f6b200b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 >> 172.21.15.32:0/3924930203 conn(0x7f6b200b6170 msgr2=0x7f6b200aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 shutdown_connections 2024-04-12T19:52:03.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:03.802+0000 7f6b25a00640 1 -- 172.21.15.32:0/3924930203 wait complete. 2024-04-12T19:52:03.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-12T19:52:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.037+0000 7fef8a400640 1 Processor -- start 2024-04-12T19:52:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.037+0000 7fef8a400640 1 -- start start 2024-04-12T19:52:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.039+0000 7fef8a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.039+0000 7fef8a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef84163300 0x7fef840d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.039+0000 7fef8a400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8415f8f0 con 0x7fef8415c600 2024-04-12T19:52:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.039+0000 7fef8a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fef8415fbd0 con 0x7fef84163300 2024-04-12T19:52:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.039+0000 7fef8a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef8415feb0 con 0x7fef840d3e20 2024-04-12T19:52:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.040+0000 7fef83e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.040+0000 7fef89a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef84163300 0x7fef840d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.040+0000 7fef83e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:48256/0 (socket says 172.21.15.32:48256) 2024-04-12T19:52:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.040+0000 7fef83e00640 1 -- 172.21.15.32:0/4049252294 learned_addr learned my addr 172.21.15.32:0/4049252294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.041+0000 7fef83e00640 1 -- 172.21.15.32:0/4049252294 >> v2:172.21.15.169:3300/0 conn(0x7fef84163300 msgr2=0x7fef840d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.041+0000 7fef83e00640 1 --2- 172.21.15.32:0/4049252294 >> v2:172.21.15.169:3300/0 conn(0x7fef84163300 0x7fef840d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.041+0000 7fef83e00640 1 -- 172.21.15.32:0/4049252294 >> v1:172.21.15.169:6789/0 conn(0x7fef8415c600 legacy=0x7fef8415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.041+0000 7fef83e00640 1 -- 172.21.15.32:0/4049252294 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef841628a0 con 0x7fef840d3e20 2024-04-12T19:52:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.042+0000 7fef83e00640 1 --2- 172.21.15.32:0/4049252294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6db58b479f46c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:04.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef83400640 1 -- 172.21.15.32:0/4049252294 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70034070 con 0x7fef840d3e20 2024-04-12T19:52:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef83400640 1 -- 172.21.15.32:0/4049252294 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef70030840 con 0x7fef840d3e20 2024-04-12T19:52:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef83400640 1 -- 172.21.15.32:0/4049252294 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70030c10 con 0x7fef840d3e20 2024-04-12T19:52:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 -- 172.21.15.32:0/4049252294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 msgr2=0x7fef84162dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 --2- 172.21.15.32:0/4049252294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef70009870 tx=0x7fef7002b980 comp rx=0 tx=0).stop 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 -- 172.21.15.32:0/4049252294 shutdown_connections 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 --2- 172.21.15.32:0/4049252294 >> v2:172.21.15.169:3300/0 conn(0x7fef84163300 0x7fef840d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 --2- 172.21.15.32:0/4049252294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef840d3e20 0x7fef84162dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 -- 172.21.15.32:0/4049252294 >> 172.21.15.32:0/4049252294 conn(0x7fef84073660 msgr2=0x7fef840d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.043+0000 7fef8a400640 1 -- 172.21.15.32:0/4049252294 shutdown_connections 2024-04-12T19:52:04.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.044+0000 7fef8a400640 1 -- 172.21.15.32:0/4049252294 wait complete. 2024-04-12T19:52:04.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.046+0000 7fef8a400640 1 Processor -- start 2024-04-12T19:52:04.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.046+0000 7fef8a400640 1 -- start start 2024-04-12T19:52:04.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8415c600 0x7fef840cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef84163300 0x7fef840ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef840d2db0 con 0x7fef840d3e20 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef840cfee0 con 0x7fef8415c600 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.047+0000 7fef8a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef840d2f30 con 0x7fef84163300 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51324/0 (socket says 172.21.15.32:51324) 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 -- 172.21.15.32:0/2422790826 learned_addr learned my addr 172.21.15.32:0/2422790826 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89a00640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef84163300 0x7fef840ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef83e00640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8415c600 0x7fef840cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 -- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef84163300 msgr2=0x7fef840ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef84163300 0x7fef840ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 -- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8415c600 msgr2=0x7fef840cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8415c600 0x7fef840cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.048+0000 7fef89000640 1 -- 172.21.15.32:0/2422790826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8415bb10 con 0x7fef840d3e20 2024-04-12T19:52:04.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef89000640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c400 tx=0x7fef7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:04.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740052d0 con 0x7fef840d3e20 2024-04-12T19:52:04.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef74005490 con 0x7fef840d3e20 2024-04-12T19:52:04.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74007260 con 0x7fef840d3e20 2024-04-12T19:52:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef840c9920 con 0x7fef840d3e20 2024-04-12T19:52:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.049+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef840c9ee0 con 0x7fef840d3e20 2024-04-12T19:52:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.051+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef840755c0 con 0x7fef840d3e20 2024-04-12T19:52:04.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.051+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef74007400 con 0x7fef840d3e20 2024-04-12T19:52:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.054+0000 7fef80c00640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 0x7fef6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.054+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7fef7408b050 con 0x7fef840d3e20 2024-04-12T19:52:04.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.054+0000 7fef83e00640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 0x7fef6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:04.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.055+0000 7fef83e00640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 0x7fef6406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef70036000 tx=0x7fef7003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.061+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef740594b0 con 0x7fef840d3e20 2024-04-12T19:52:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.268+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef840d1e70 con 0x7fef840d3e20 2024-04-12T19:52:04.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.823+0000 7fef80c00640 1 -- 172.21.15.32:0/2422790826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7fef74058db0 con 0x7fef840d3e20 2024-04-12T19:52:04.823 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-12T19:52:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.827+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 msgr2=0x7fef6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.827+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 0x7fef6406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef70036000 tx=0x7fef7003b040 comp rx=0 tx=0).stop 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.827+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 msgr2=0x7fef8415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.827+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c400 tx=0x7fef7400c8c0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 shutdown_connections 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fef6406c8f0 0x7fef6406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef84163300 0x7fef840ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8415c600 0x7fef840cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 --2- 172.21.15.32:0/2422790826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef840d3e20 0x7fef8415b5d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 >> 172.21.15.32:0/2422790826 conn(0x7fef84073660 msgr2=0x7fef8406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 shutdown_connections 2024-04-12T19:52:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:04.828+0000 7fef8a400640 1 -- 172.21.15.32:0/2422790826 wait complete. 2024-04-12T19:52:04.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-12T19:52:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.072+0000 7f3615e00640 1 Processor -- start 2024-04-12T19:52:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.072+0000 7f3615e00640 1 -- start start 2024-04-12T19:52:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f3615e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36101478b0 0x7f36100afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f3615e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f361006f920 0x7f36101626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f3615e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36100aebd0 con 0x7f3610147c90 2024-04-12T19:52:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f3615e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f36100b0130 con 0x7f361006f920 2024-04-12T19:52:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f3615e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f36100aee50 con 0x7f36101478b0 2024-04-12T19:52:05.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f360fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3610147c90 0x7f361006f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37338/0 (socket says 172.21.15.32:37338) 2024-04-12T19:52:05.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.074+0000 7f360fe00640 1 -- 172.21.15.32:0/3374306140 learned_addr learned my addr 172.21.15.32:0/3374306140 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:05.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3615400640 1 --2- 172.21.15.32:0/3374306140 >> v2:172.21.15.32:3300/0 conn(0x7f361006f920 0x7f36101626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3614a00640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36101478b0 0x7f36100afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627045436 0 0) 0x7f35fc001110 con 0x7f3610147c90 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35f8003520 con 0x7f3610147c90 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3615400640 1 -- 172.21.15.32:0/3374306140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36101478b0 msgr2=0x7f36100afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3615400640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36101478b0 0x7f36100afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634912721 0 0) 0x7f35f8003520 con 0x7f3610147c90 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3615400640 1 -- 172.21.15.32:0/3374306140 >> v1:172.21.15.32:6789/0 conn(0x7f3610147c90 legacy=0x7f361006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.075+0000 7f3615400640 1 -- 172.21.15.32:0/3374306140 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36100af110 con 0x7f361006f920 2024-04-12T19:52:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.076+0000 7f3615400640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f361006f920 0x7f36101626c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f360802bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c3dbb09a0c2cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.077+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608039070 con 0x7f361006f920 2024-04-12T19:52:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.077+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360802c8a0 con 0x7f361006f920 2024-04-12T19:52:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.077+0000 7f360f400640 1 -- 172.21.15.32:0/3374306140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc001860 con 0x7f361006f920 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.077+0000 7f3615e00640 1 -- 172.21.15.32:0/3374306140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f361006f920 msgr2=0x7f36101626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.077+0000 7f3615e00640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f361006f920 0x7f36101626c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3608009d90 tx=0x7f360802bd70 comp rx=0 tx=0).stop 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 -- 172.21.15.32:0/3374306140 shutdown_connections 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f361006f920 0x7f36101626c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 --2- 172.21.15.32:0/3374306140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36101478b0 0x7f36100afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 -- 172.21.15.32:0/3374306140 >> 172.21.15.32:0/3374306140 conn(0x7f361006b930 msgr2=0x7f361006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 -- 172.21.15.32:0/3374306140 shutdown_connections 2024-04-12T19:52:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.078+0000 7f3615e00640 1 -- 172.21.15.32:0/3374306140 wait complete. 2024-04-12T19:52:05.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.079+0000 7f3615e00640 1 Processor -- start 2024-04-12T19:52:05.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.080+0000 7f3615e00640 1 -- start start 2024-04-12T19:52:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.080+0000 7f3615e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f361006f920 0x7f36100cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.080+0000 7f3615e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f3615e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3610147c90 0x7f361015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f3615e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3610162c00 con 0x7f36101478b0 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f3615e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36100af6d0 con 0x7f3610147c90 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f3615e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3610162d80 con 0x7f361006f920 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51344/0 (socket says 172.21.15.32:51344) 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 -- 172.21.15.32:0/1691616577 learned_addr learned my addr 172.21.15.32:0/1691616577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 -- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f361006f920 msgr2=0x7f36100cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f361006f920 0x7f36100cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 -- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3610147c90 msgr2=0x7f361015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3610147c90 0x7f361015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.081+0000 7f360fe00640 1 -- 172.21.15.32:0/1691616577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36100cc9d0 con 0x7f36101478b0 2024-04-12T19:52:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f360fe00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c580 tx=0x7f35fc00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc018070 con 0x7f36101478b0 2024-04-12T19:52:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35fc007580 con 0x7f36101478b0 2024-04-12T19:52:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc004a20 con 0x7f36101478b0 2024-04-12T19:52:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36100ccc90 con 0x7f36101478b0 2024-04-12T19:52:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.082+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36100cf820 con 0x7f36101478b0 2024-04-12T19:52:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.083+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361006c8c0 con 0x7f36101478b0 2024-04-12T19:52:05.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.084+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35fc004bc0 con 0x7f36101478b0 2024-04-12T19:52:05.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.085+0000 7f360cc00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 0x7f35f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.085+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f35fc08abd0 con 0x7f36101478b0 2024-04-12T19:52:05.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.085+0000 7f3614a00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 0x7f35f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.086+0000 7f3614a00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 0x7f35f006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3604006a90 tx=0x7f3604006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.090+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36100aebd0 con 0x7f36101478b0 2024-04-12T19:52:05.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.304+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f36100755c0 con 0x7f36101478b0 2024-04-12T19:52:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.311+0000 7f360cc00640 1 -- 172.21.15.32:0/1691616577 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f35fc0592f0 con 0x7f36101478b0 2024-04-12T19:52:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.315+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 msgr2=0x7f35f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.315+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 0x7f35f006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3604006a90 tx=0x7f3604006a20 comp rx=0 tx=0).stop 2024-04-12T19:52:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.315+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 msgr2=0x7f36101e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c580 tx=0x7f35fc00ca40 comp rx=0 tx=0).stop 2024-04-12T19:52:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 shutdown_connections 2024-04-12T19:52:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f35f006c8f0 0x7f35f006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3610147c90 0x7f361015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36101478b0 0x7f36101e8ae0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 --2- 172.21.15.32:0/1691616577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f361006f920 0x7f36100cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 >> 172.21.15.32:0/1691616577 conn(0x7f361006b930 msgr2=0x7f36100b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 shutdown_connections 2024-04-12T19:52:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.316+0000 7f3615e00640 1 -- 172.21.15.32:0/1691616577 wait complete. 2024-04-12T19:52:05.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_flush.TestFlush.test_flush' 2024-04-12T19:52:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.555+0000 7fa013e00640 1 Processor -- start 2024-04-12T19:52:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.556+0000 7fa013e00640 1 -- start start 2024-04-12T19:52:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.557+0000 7fa013e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140cee00 0x7fa01406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.557+0000 7fa013e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa014071f50 0x7fa0140723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.557+0000 7fa013e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01415dbb0 con 0x7fa01415cbe0 2024-04-12T19:52:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.557+0000 7fa013e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa01406f4f0 con 0x7fa0140cee00 2024-04-12T19:52:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.557+0000 7fa013e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0140728f0 con 0x7fa014071f50 2024-04-12T19:52:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140cee00 0x7fa01406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140cee00 0x7fa01406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51352/0 (socket says 172.21.15.32:51352) 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 -- 172.21.15.32:0/3986017624 learned_addr learned my addr 172.21.15.32:0/3986017624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 -- 172.21.15.32:0/3986017624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa014071f50 msgr2=0x7fa0140723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 --2- 172.21.15.32:0/3986017624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa014071f50 0x7fa0140723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 -- 172.21.15.32:0/3986017624 >> v1:172.21.15.169:6789/0 conn(0x7fa01415cbe0 legacy=0x7fa01415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.558+0000 7fa012a00640 1 -- 172.21.15.32:0/3986017624 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01415d8d0 con 0x7fa0140cee00 2024-04-12T19:52:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.559+0000 7fa012a00640 1 --2- 172.21.15.32:0/3986017624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140cee00 0x7fa01406e8c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098a0 tx=0x7fa00402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56761132e8ef7107 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.560+0000 7fa012000640 1 -- 172.21.15.32:0/3986017624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004035020 con 0x7fa0140cee00 2024-04-12T19:52:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.560+0000 7fa012000640 1 -- 172.21.15.32:0/3986017624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa004031640 con 0x7fa0140cee00 2024-04-12T19:52:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.560+0000 7fa012000640 1 -- 172.21.15.32:0/3986017624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00402d040 con 0x7fa0140cee00 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 -- 172.21.15.32:0/3986017624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140cee00 msgr2=0x7fa01406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 --2- 172.21.15.32:0/3986017624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140cee00 0x7fa01406e8c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098a0 tx=0x7fa00402b8f0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 -- 172.21.15.32:0/3986017624 shutdown_connections 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 --2- 172.21.15.32:0/3986017624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa014071f50 0x7fa0140723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 --2- 172.21.15.32:0/3986017624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140cee00 0x7fa01406e8c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 -- 172.21.15.32:0/3986017624 >> 172.21.15.32:0/3986017624 conn(0x7fa01406b930 msgr2=0x7fa01406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 -- 172.21.15.32:0/3986017624 shutdown_connections 2024-04-12T19:52:05.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.561+0000 7fa013e00640 1 -- 172.21.15.32:0/3986017624 wait complete. 2024-04-12T19:52:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.563+0000 7fa013e00640 1 Processor -- start 2024-04-12T19:52:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.563+0000 7fa013e00640 1 -- start start 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa014071f50 0x7fa014145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 0x7fa01414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406e3a0 con 0x7fa01415cbe0 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406e0c0 con 0x7fa014071f50 2024-04-12T19:52:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa01415dda0 con 0x7fa0140cee00 2024-04-12T19:52:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51366/0 (socket says 172.21.15.32:51366) 2024-04-12T19:52:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa013400640 1 -- 172.21.15.32:0/4196739375 learned_addr learned my addr 172.21.15.32:0/4196739375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa012a00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa014071f50 0x7fa014145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.564+0000 7fa00a000640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 0x7fa01414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 -- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 msgr2=0x7fa01414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 0x7fa01414ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 -- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa014071f50 msgr2=0x7fa014145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa014071f50 0x7fa014145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 -- 172.21.15.32:0/4196739375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa014151360 con 0x7fa01415cbe0 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa00a000640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 0x7fa01414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013400640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00b970 tx=0x7fa00c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c019070 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa00c00cb40 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c014440 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa014151640 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.565+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa014151c00 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.567+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014146350 con 0x7fa01415cbe0 2024-04-12T19:52:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.568+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa00c01c070 con 0x7fa01415cbe0 2024-04-12T19:52:05.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.568+0000 7fa00be00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 0x7f9ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.569+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa00c08b510 con 0x7fa01415cbe0 2024-04-12T19:52:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.569+0000 7fa012a00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 0x7f9ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:05.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.570+0000 7fa012a00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 0x7f9ff006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa004036000 tx=0x7fa00403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:05.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.573+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa00c059ba0 con 0x7fa01415cbe0 2024-04-12T19:52:05.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:05.855+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]} v 0) -- 0x7fa014075540 con 0x7fa01415cbe0 2024-04-12T19:52:06.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.829+0000 7fa00be00640 1 -- 172.21.15.32:0/4196739375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_flush.TestFlush.test_flush"]}]=0 v119) ==== 123+0+0 (secure 0 0 0) 0x7fa00c0594a0 con 0x7fa01415cbe0 2024-04-12T19:52:06.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.832+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 msgr2=0x7f9ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.832+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 0x7f9ff006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa004036000 tx=0x7fa00403c040 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.832+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 msgr2=0x7fa014150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.832+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00b970 tx=0x7fa00c00be30 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 shutdown_connections 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f9ff006c8f0 0x7f9ff006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01415cbe0 0x7fa014150690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0140cee00 0x7fa01414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 --2- 172.21.15.32:0/4196739375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa014071f50 0x7fa014145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:06.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 >> 172.21.15.32:0/4196739375 conn(0x7fa01406b930 msgr2=0x7fa0140b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 shutdown_connections 2024-04-12T19:52:06.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:06.833+0000 7fa013e00640 1 -- 172.21.15.32:0/4196739375 wait complete. 2024-04-12T19:52:06.857 INFO:tasks.cephfs_test_runner:test_flush (tasks.cephfs.test_flush.TestFlush) ... ERROR 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner: 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner:ERROR: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/test_flush.py", line 47, in test_flush 2024-04-12T19:52:06.858 INFO:tasks.cephfs_test_runner: summary_output = self.fs.journal_tool(["event", "get", "summary"], 0) 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/filesystem.py", line 1708, in journal_tool 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: return self._run_tool("cephfs-journal-tool", args, fs_rank, quiet) 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/filesystem.py", line 1687, in _run_tool 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: r = self.tool_remote.sh(script=base_args + args, stdout=StringIO()).strip() 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: proc = self.run(**kwargs) 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 255: 'cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary' 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner: 2024-04-12T19:52:06.859 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:Ran 1 test in 62.280s 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:ERROR: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/test_flush.py", line 47, in test_flush 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: summary_output = self.fs.journal_tool(["event", "get", "summary"], 0) 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/filesystem.py", line 1708, in journal_tool 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: return self._run_tool("cephfs-journal-tool", args, fs_rank, quiet) 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/filesystem.py", line 1687, in _run_tool 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: r = self.tool_remote.sh(script=base_args + args, stdout=StringIO()).strip() 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh 2024-04-12T19:52:06.860 INFO:tasks.cephfs_test_runner: proc = self.run(**kwargs) 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 255: 'cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event get summary' 2024-04-12T19:52:06.861 INFO:tasks.cephfs_test_runner: 2024-04-12T19:52:06.861 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-04-12T19:52:07.129 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=755c47263f0843efbcd7c4e919486483 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flush (tasks.cephfs.test_flush.TestFlush) 2024-04-12T19:52:07.131 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-12T19:52:07.140 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-12T19:52:07.157 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-12T19:52:07.157 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:52:07.157 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:52:07.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053546,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-12T19:52:07.222 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-12T19:52:07.222 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:52:07.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053546,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-12T19:52:07.280 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:52:07.280 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:52:07.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-12T19:52:07.339 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-12T19:52:07.339 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T19:52:07.393 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052773,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-12T19:52:07.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.394 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:52:07.454 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-12T19:52:07.454 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-12T19:52:07.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T19:52:07.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-12T19:52:07.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-12T19:52:07.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.781 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-12T19:52:07.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-12T19:52:07.961 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-12T19:52:07.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:07.965 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:52:08.023 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-12T19:52:08.024 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-12T19:52:08.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:08.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T19:52:08.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:08.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-12T19:52:08.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:08.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-12T19:52:08.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:08.332 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-12T19:52:08.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T19:52:08.375 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-12T19:52:08.509 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-12T19:52:08.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:08.513 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-12T19:52:08.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-12T19:52:08.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:52:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.819+0000 7f7d71e00640 1 Processor -- start 2024-04-12T19:52:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.819+0000 7f7d71e00640 1 -- start start 2024-04-12T19:52:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d71e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d6c0cee00 0x7f7d6c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d71e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c15dbb0 con 0x7f7d6c071f50 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d71e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6c06f4f0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d71e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0728f0 con 0x7f7d6c15d2f0 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d6be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d6c071f50 0x7f7d6c15cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37346/0 (socket says 172.21.15.32:37346) 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d70a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d6c0cee00 0x7f7d6c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.821+0000 7f7d6be00640 1 -- 172.21.15.32:0/327516497 learned_addr learned my addr 172.21.15.32:0/327516497 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d71400640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d6b400640 1 -- 172.21.15.32:0/327516497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3206639964 0 0) 0x7f7d6c15dbb0 con 0x7f7d6c071f50 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d70a00640 1 -- 172.21.15.32:0/327516497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 msgr2=0x7f7d6c0723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d70a00640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c0723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d70a00640 1 -- 172.21.15.32:0/327516497 >> v1:172.21.15.32:6789/0 conn(0x7f7d6c071f50 legacy=0x7f7d6c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.822+0000 7f7d70a00640 1 -- 172.21.15.32:0/327516497 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c15d8d0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d70a00640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c06e8c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097a0 tx=0x7f7d6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac82212074a6282 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:08.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d6b400640 1 -- 172.21.15.32:0/327516497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60035020 con 0x7f7d6c0cee00 2024-04-12T19:52:08.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d6b400640 1 -- 172.21.15.32:0/327516497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60031830 con 0x7f7d6c0cee00 2024-04-12T19:52:08.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d6b400640 1 -- 172.21.15.32:0/327516497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60031c00 con 0x7f7d6c0cee00 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d71e00640 1 -- 172.21.15.32:0/327516497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 msgr2=0x7f7d6c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.824+0000 7f7d71e00640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c06e8c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097a0 tx=0x7f7d6002b8d0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 -- 172.21.15.32:0/327516497 shutdown_connections 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 --2- 172.21.15.32:0/327516497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c06e8c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 -- 172.21.15.32:0/327516497 >> 172.21.15.32:0/327516497 conn(0x7f7d6c06b930 msgr2=0x7f7d6c06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 -- 172.21.15.32:0/327516497 shutdown_connections 2024-04-12T19:52:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.825+0000 7f7d71e00640 1 -- 172.21.15.32:0/327516497 wait complete. 2024-04-12T19:52:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.826+0000 7f7d71e00640 1 Processor -- start 2024-04-12T19:52:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.827+0000 7f7d71e00640 1 -- start start 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 0x7f7d6c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c06e3a0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c06e0c0 con 0x7f7d6c071f50 2024-04-12T19:52:08.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c15dda0 con 0x7f7d6c15d2f0 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51376/0 (socket says 172.21.15.32:51376) 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 -- 172.21.15.32:0/3187441942 learned_addr learned my addr 172.21.15.32:0/3187441942 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d70a00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 0x7f7d6c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 -- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 msgr2=0x7f7d6c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d71400640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c15f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 -- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 msgr2=0x7f7d6c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:08.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 0x7f7d6c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d6be00640 1 -- 172.21.15.32:0/3187441942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c1f6010 con 0x7f7d6c0cee00 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.828+0000 7f7d70a00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 0x7f7d6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d6be00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c4d0 tx=0x7f7d5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c018070 con 0x7f7d6c0cee00 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c007500 con 0x7f7d6c0cee00 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004a20 con 0x7f7d6c0cee00 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c1f62f0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.829+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1f68b0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.830+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c14b750 con 0x7f7d6c0cee00 2024-04-12T19:52:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.831+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d5c01b070 con 0x7f7d6c0cee00 2024-04-12T19:52:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.832+0000 7f7d68c00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 0x7f7d5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.832+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f7d5c08aba0 con 0x7f7d6c0cee00 2024-04-12T19:52:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.833+0000 7f7d70a00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 0x7f7d5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.833+0000 7f7d70a00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 0x7f7d5006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d60036000 tx=0x7f7d60008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:08.837+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d5c0592c0 con 0x7f7d6c0cee00 2024-04-12T19:52:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.040+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d6c15a2d0 con 0x7f7d5006c8f0 2024-04-12T19:52:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.042+0000 7f7d68c00640 1 -- 172.21.15.32:0/3187441942 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f7d6c15a2d0 con 0x7f7d5006c8f0 2024-04-12T19:52:09.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:09.043 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:52:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 msgr2=0x7f7d5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 0x7f7d5006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d60036000 tx=0x7f7d60008030 comp rx=0 tx=0).stop 2024-04-12T19:52:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 msgr2=0x7f7d6c14e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c4d0 tx=0x7f7d5c00c990 comp rx=0 tx=0).stop 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 shutdown_connections 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7d5006c8f0 0x7f7d5006edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c15d2f0 0x7f7d6c15f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6c0cee00 0x7f7d6c14e910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 --2- 172.21.15.32:0/3187441942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c071f50 0x7f7d6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 >> 172.21.15.32:0/3187441942 conn(0x7f7d6c06b930 msgr2=0x7f7d6c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 shutdown_connections 2024-04-12T19:52:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.045+0000 7f7d71e00640 1 -- 172.21.15.32:0/3187441942 wait complete. 2024-04-12T19:52:09.066 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-12T19:52:07.657990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241532,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733188,"statfs":{"total":773094113280,"available":772846784512,"internally_reserved":0,"allocated":4661248,"data_stored":2533704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15383,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-76,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002111"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-12T19:52:05.309318+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:52:05.309318+0000","last_peered":"2024-04-12T19:52:05.309318+0000","last_clean":"2024-04-12T19:52:05.309318+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:52:05.309318+0000","last_undegraded":"2024-04-12T19:52:05.309318+0000","last_fullsized":"2024-04-12T19:52:05.309318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:52:09.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:52:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.290+0000 7ff73e600640 1 Processor -- start 2024-04-12T19:52:09.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.290+0000 7ff73e600640 1 -- start start 2024-04-12T19:52:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff73e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff73806f4f0 0x7ff7381663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff73e600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7380cdfb0 con 0x7ff73815b860 2024-04-12T19:52:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff73e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff73815f2b0 con 0x7ff73815b480 2024-04-12T19:52:09.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff73e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff73815f060 con 0x7ff73806f4f0 2024-04-12T19:52:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff737e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff73815b860 0x7ff73806ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37358/0 (socket says 172.21.15.32:37358) 2024-04-12T19:52:09.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.292+0000 7ff737e00640 1 -- 172.21.15.32:0/3079524978 learned_addr learned my addr 172.21.15.32:0/3079524978 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff73dc00640 1 --2- 172.21.15.32:0/3079524978 >> v2:172.21.15.169:3300/0 conn(0x7ff73806f4f0 0x7ff7381663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180795448 0 0) 0x7ff7380cdfb0 con 0x7ff73815b860 2024-04-12T19:52:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff720003520 con 0x7ff73815b860 2024-04-12T19:52:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1371322425 0 0) 0x7ff720003520 con 0x7ff73815b860 2024-04-12T19:52:09.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 >> v2:172.21.15.169:3300/0 conn(0x7ff73806f4f0 msgr2=0x7ff7381663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 --2- 172.21.15.32:0/3079524978 >> v2:172.21.15.169:3300/0 conn(0x7ff73806f4f0 0x7ff7381663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 >> v1:172.21.15.169:6789/0 conn(0x7ff73815b480 legacy=0x7ff7380cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73815f590 con 0x7ff73815b860 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7ff724001670 con 0x7ff73815b860 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 >> v1:172.21.15.32:6789/0 conn(0x7ff73815b860 legacy=0x7ff73806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.293+0000 7ff737400640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff720004800 0x7ff720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737400640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7200051c0 0x7ff72000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff73815f060 con 0x7ff7200039b0 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff72000c060 con 0x7ff7200051c0 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff72000c260 con 0x7ff720004800 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73dc00640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7200051c0 0x7ff72000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737e00640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff720004800 0x7ff720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 -- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff720004800 msgr2=0x7ff720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff720004800 0x7ff720004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 -- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7200051c0 msgr2=0x7ff72000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7200051c0 0x7ff72000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73d200640 1 -- 172.21.15.32:0/3079524978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff720003520 con 0x7ff7200039b0 2024-04-12T19:52:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff737e00640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff720004800 0x7ff720004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73dc00640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7200051c0 0x7ff72000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.294+0000 7ff73dc00640 1 -- 172.21.15.32:0/3079524978 reap_dead start 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff73d200640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001750 tx=0x7ff720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc80e1f210e38836 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c001370 con 0x7ff7200039b0 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c0052d0 con 0x7ff7200039b0 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff737400640 1 -- 172.21.15.32:0/3079524978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c003830 con 0x7ff7200039b0 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff73e600640 1 -- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 msgr2=0x7ff720003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.296+0000 7ff73e600640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff72c001750 tx=0x7ff720005830 comp rx=0 tx=0).stop 2024-04-12T19:52:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.297+0000 7ff73e600640 1 -- 172.21.15.32:0/3079524978 shutdown_connections 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.297+0000 7ff73e600640 1 --2- 172.21.15.32:0/3079524978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.297+0000 7ff73e600640 1 -- 172.21.15.32:0/3079524978 >> 172.21.15.32:0/3079524978 conn(0x7ff73806b930 msgr2=0x7ff73806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.297+0000 7ff73e600640 1 -- 172.21.15.32:0/3079524978 shutdown_connections 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.297+0000 7ff73e600640 1 -- 172.21.15.32:0/3079524978 wait complete. 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.298+0000 7ff73e600640 1 Processor -- start 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.299+0000 7ff73e600640 1 -- start start 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.299+0000 7ff73e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.299+0000 7ff73e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff73815da30 0x7ff738159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff738149b20 0x7ff738149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73e600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff73815fb50 con 0x7ff7200039b0 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff73815fec0 con 0x7ff738149b20 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff7380cd550 con 0x7ff73815da30 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51394/0 (socket says 172.21.15.32:51394) 2024-04-12T19:52:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 -- 172.21.15.32:0/3502262583 learned_addr learned my addr 172.21.15.32:0/3502262583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73dc00640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff738149b20 0x7ff738149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 -- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff73815da30 msgr2=0x7ff738159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff73815da30 0x7ff738159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 -- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff738149b20 msgr2=0x7ff738149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff738149b20 0x7ff738149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.300+0000 7ff73d200640 1 -- 172.21.15.32:0/3502262583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73815c5a0 con 0x7ff7200039b0 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff73d200640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff72c000c00 tx=0x7ff72c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c041020 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c005480 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c007a30 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73815c880 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.301+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7380c9aa0 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.302+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7380755c0 con 0x7ff7200039b0 2024-04-12T19:52:09.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.303+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff72c044020 con 0x7ff7200039b0 2024-04-12T19:52:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.304+0000 7ff734c00640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 0x7ff71806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.304+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff72c0ad210 con 0x7ff7200039b0 2024-04-12T19:52:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.305+0000 7ff737e00640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 0x7ff71806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.305+0000 7ff737e00640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 0x7ff71806ee00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff724000ff0 tx=0x7ff724009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.309+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff72c04b030 con 0x7ff7200039b0 2024-04-12T19:52:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.511+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7380b58e0 con 0x7ff71806c940 2024-04-12T19:52:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.513+0000 7ff734c00640 1 -- 172.21.15.32:0/3502262583 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7ff7380b58e0 con 0x7ff71806c940 2024-04-12T19:52:09.513 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:52:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.515+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 msgr2=0x7ff71806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.515+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 0x7ff71806ee00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff724000ff0 tx=0x7ff724009040 comp rx=0 tx=0).stop 2024-04-12T19:52:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.515+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 msgr2=0x7ff73815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.515+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff72c000c00 tx=0x7ff72c001120 comp rx=0 tx=0).stop 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 shutdown_connections 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7ff71806c940 0x7ff71806ee00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff738149b20 0x7ff738149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff73815da30 0x7ff738159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 --2- 172.21.15.32:0/3502262583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7200039b0 0x7ff73815bc70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 >> 172.21.15.32:0/3502262583 conn(0x7ff73806b930 msgr2=0x7ff7380b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 shutdown_connections 2024-04-12T19:52:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.516+0000 7ff73e600640 1 -- 172.21.15.32:0/3502262583 wait complete. 2024-04-12T19:52:09.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-04-12T19:52:07.657990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241532,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733188,"statfs":{"total":773094113280,"available":772846784512,"internally_reserved":0,"allocated":4661248,"data_stored":2533704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15383,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-76,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002111"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-12T19:52:05.309318+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:52:05.309318+0000","last_peered":"2024-04-12T19:52:05.309318+0000","last_clean":"2024-04-12T19:52:05.309318+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:52:05.309318+0000","last_undegraded":"2024-04-12T19:52:05.309318+0000","last_fullsized":"2024-04-12T19:52:05.309318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:52:09.535 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-12T19:52:09.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:52:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.750+0000 7f106fe00640 1 Processor -- start 2024-04-12T19:52:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.750+0000 7f106fe00640 1 -- start start 2024-04-12T19:52:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f10700cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f107015b860 0x7f10700b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106fe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107015f2b0 con 0x7f10700cdde0 2024-04-12T19:52:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f107015f060 con 0x7f107015b860 2024-04-12T19:52:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f107015fec0 con 0x7f107015b480 2024-04-12T19:52:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106e000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10700cdde0 0x7f107006ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37372/0 (socket says 172.21.15.32:37372) 2024-04-12T19:52:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.752+0000 7f106e000640 1 -- 172.21.15.32:0/1387929407 learned_addr learned my addr 172.21.15.32:0/1387929407 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106ea00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f10700cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106f400640 1 --2- 172.21.15.32:0/1387929407 >> v2:172.21.15.169:3300/0 conn(0x7f107015b860 0x7f10700b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996586394 0 0) 0x7f107015f2b0 con 0x7f10700cdde0 2024-04-12T19:52:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1054003520 con 0x7f10700cdde0 2024-04-12T19:52:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2013876497 0 0) 0x7f1054003520 con 0x7f10700cdde0 2024-04-12T19:52:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 msgr2=0x7f10700cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f10700cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 >> v2:172.21.15.169:3300/0 conn(0x7f107015b860 msgr2=0x7f10700b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 --2- 172.21.15.32:0/1387929407 >> v2:172.21.15.169:3300/0 conn(0x7f107015b860 0x7f10700b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107015f590 con 0x7f10700cdde0 2024-04-12T19:52:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f1058001670 con 0x7f10700cdde0 2024-04-12T19:52:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.753+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 >> v1:172.21.15.32:6789/0 conn(0x7f10700cdde0 legacy=0x7f107006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10540039b0 0x7f1054003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f1054004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1054004d40 0x7f10540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f107015f060 con 0x7f10540039b0 2024-04-12T19:52:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f107015fec0 con 0x7f1054004d40 2024-04-12T19:52:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f105400dbd0 con 0x7f107015b480 2024-04-12T19:52:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10540039b0 0x7f1054003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106e000640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f1054004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 -- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 msgr2=0x7f1054004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f1054004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 -- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1054004d40 msgr2=0x7f10540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1054004d40 0x7f10540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106ea00640 1 -- 172.21.15.32:0/1387929407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1054003520 con 0x7f10540039b0 2024-04-12T19:52:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.754+0000 7f106e000640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107015b480 0x7f1054004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106ea00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10540039b0 0x7f1054003df0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1060009ac0 tx=0x7f1054005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b507150c73d441fe server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060039070 con 0x7f10540039b0 2024-04-12T19:52:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106002c6b0 con 0x7f10540039b0 2024-04-12T19:52:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106d600640 1 -- 172.21.15.32:0/1387929407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060009690 con 0x7f10540039b0 2024-04-12T19:52:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106fe00640 1 -- 172.21.15.32:0/1387929407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10540039b0 msgr2=0x7f1054003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106fe00640 1 --2- 172.21.15.32:0/1387929407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10540039b0 0x7f1054003df0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1060009ac0 tx=0x7f1054005830 comp rx=0 tx=0).stop 2024-04-12T19:52:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.756+0000 7f106f400640 1 -- 172.21.15.32:0/1387929407 reap_dead start 2024-04-12T19:52:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.757+0000 7f106fe00640 1 -- 172.21.15.32:0/1387929407 shutdown_connections 2024-04-12T19:52:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.757+0000 7f106fe00640 1 -- 172.21.15.32:0/1387929407 >> 172.21.15.32:0/1387929407 conn(0x7f107006b930 msgr2=0x7f107006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.757+0000 7f106fe00640 1 -- 172.21.15.32:0/1387929407 shutdown_connections 2024-04-12T19:52:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.757+0000 7f106fe00640 1 -- 172.21.15.32:0/1387929407 wait complete. 2024-04-12T19:52:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.758+0000 7f106fe00640 1 Processor -- start 2024-04-12T19:52:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.759+0000 7f106fe00640 1 -- start start 2024-04-12T19:52:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.759+0000 7f106fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10540039b0 0x7f1070147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.759+0000 7f106fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 0x7f107015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10700cd550 con 0x7f1070152270 2024-04-12T19:52:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f107015fb50 con 0x7f107015d640 2024-04-12T19:52:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f107006f4f0 con 0x7f10540039b0 2024-04-12T19:52:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51424/0 (socket says 172.21.15.32:51424) 2024-04-12T19:52:09.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 -- 172.21.15.32:0/2537728447 learned_addr learned my addr 172.21.15.32:0/2537728447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106ea00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10540039b0 0x7f1070147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106f400640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 0x7f107015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 -- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10540039b0 msgr2=0x7f1070147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10540039b0 0x7f1070147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 -- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 msgr2=0x7f107015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 0x7f107015daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106e000640 1 -- 172.21.15.32:0/2537728447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10700c9a00 con 0x7f1070152270 2024-04-12T19:52:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.760+0000 7f106f400640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 0x7f107015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f106e000640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1058000ff0 tx=0x7f105800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1058019070 con 0x7f1070152270 2024-04-12T19:52:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10580074e0 con 0x7f1070152270 2024-04-12T19:52:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10700c9ce0 con 0x7f1070152270 2024-04-12T19:52:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580049e0 con 0x7f1070152270 2024-04-12T19:52:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.761+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10700ca2a0 con 0x7f1070152270 2024-04-12T19:52:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.763+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10700755c0 con 0x7f1070152270 2024-04-12T19:52:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.763+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f105801c070 con 0x7f1070152270 2024-04-12T19:52:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.765+0000 7f1066a00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 0x7f104c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.765+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f105808b230 con 0x7f1070152270 2024-04-12T19:52:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.765+0000 7f106ea00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 0x7f104c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.766+0000 7f106ea00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 0x7f104c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1060036000 tx=0x7f1060030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.772+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10580599f0 con 0x7f1070152270 2024-04-12T19:52:09.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.975+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10700b3930 con 0x7f104c06c8f0 2024-04-12T19:52:09.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.976+0000 7f1066a00640 1 -- 172.21.15.32:0/2537728447 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f10700b3930 con 0x7f104c06c8f0 2024-04-12T19:52:09.977 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:09.977 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:52:09.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 msgr2=0x7f104c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 0x7f104c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1060036000 tx=0x7f1060030000 comp rx=0 tx=0).stop 2024-04-12T19:52:09.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 msgr2=0x7f1070159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:09.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1058000ff0 tx=0x7f105800dc90 comp rx=0 tx=0).stop 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 shutdown_connections 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f104c06c8f0 0x7f104c06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107015d640 0x7f107015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1070152270 0x7f1070159c20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 --2- 172.21.15.32:0/2537728447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10540039b0 0x7f1070147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 >> 172.21.15.32:0/2537728447 conn(0x7f107006b930 msgr2=0x7f1070156980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.979+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 shutdown_connections 2024-04-12T19:52:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:09.980+0000 7f106fe00640 1 -- 172.21.15.32:0/2537728447 wait complete. 2024-04-12T19:52:10.001 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-12T19:52:09.658450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241532,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733188,"statfs":{"total":773094113280,"available":772846784512,"internally_reserved":0,"allocated":4661248,"data_stored":2533704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15383,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-76,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.936128"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-12T19:52:05.309318+0000","last_change":"2024-04-12T19:50:13.967693+0000","last_active":"2024-04-12T19:52:05.309318+0000","last_peered":"2024-04-12T19:52:05.309318+0000","last_clean":"2024-04-12T19:52:05.309318+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:52:05.309318+0000","last_undegraded":"2024-04-12T19:52:05.309318+0000","last_fullsized":"2024-04-12T19:52:05.309318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T19:50:12.428918+0000","last_clean_scrub_stamp":"2024-04-12T19:50:12.428918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T05:33:59.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:52:10.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T19:52:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.225+0000 7f2b43e00640 1 Processor -- start 2024-04-12T19:52:10.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.226+0000 7f2b43e00640 1 -- start start 2024-04-12T19:52:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.227+0000 7f2b43e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2b440d1c90 0x7f2b44162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.227+0000 7f2b43e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b440d0090 con 0x7f2b440cf110 2024-04-12T19:52:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.227+0000 7f2b43e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b440cfad0 con 0x7f2b44162020 2024-04-12T19:52:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.227+0000 7f2b43e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2b44163440 con 0x7f2b440d1c90 2024-04-12T19:52:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.228+0000 7f2b43400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b440cf110 0x7f2b440d3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37386/0 (socket says 172.21.15.32:37386) 2024-04-12T19:52:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.228+0000 7f2b43400640 1 -- 172.21.15.32:0/544621695 learned_addr learned my addr 172.21.15.32:0/544621695 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:10.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.228+0000 7f2b42000640 1 --2- 172.21.15.32:0/544621695 >> v2:172.21.15.169:3300/0 conn(0x7f2b440d1c90 0x7f2b44162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.228+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965095533 0 0) 0x7f2b3c001110 con 0x7f2b440cf110 2024-04-12T19:52:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.228+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b28003520 con 0x7f2b440cf110 2024-04-12T19:52:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201850856 0 0) 0x7f2b28003520 con 0x7f2b440cf110 2024-04-12T19:52:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 >> v2:172.21.15.169:3300/0 conn(0x7f2b440d1c90 msgr2=0x7f2b44162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:10.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 --2- 172.21.15.32:0/544621695 >> v2:172.21.15.169:3300/0 conn(0x7f2b440d1c90 0x7f2b44162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 >> v1:172.21.15.169:6789/0 conn(0x7f2b44162020 legacy=0x7f2b440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b440cfdb0 con 0x7f2b440cf110 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f2b3c001840 con 0x7f2b440cf110 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 >> v1:172.21.15.32:6789/0 conn(0x7f2b440cf110 legacy=0x7f2b440d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b280039b0 0x7f2b28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 0x7f2b28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b280051c0 0x7f2b2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44163440 con 0x7f2b28004800 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2800c060 con 0x7f2b280039b0 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2800c260 con 0x7f2b280051c0 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b42000640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 0x7f2b28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.229+0000 7f2b42a00640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b280039b0 0x7f2b28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b43400640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b280051c0 0x7f2b2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b42000640 1 -- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b280051c0 msgr2=0x7f2b2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b42000640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b280051c0 0x7f2b2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b42000640 1 -- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b280039b0 msgr2=0x7f2b28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b42000640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b280039b0 0x7f2b28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b42000640 1 -- 172.21.15.32:0/544621695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28003520 con 0x7f2b28004800 2024-04-12T19:52:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b43400640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b280051c0 0x7f2b2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.230+0000 7f2b43400640 1 -- 172.21.15.32:0/544621695 reap_dead start 2024-04-12T19:52:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.231+0000 7f2b42000640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 0x7f2b28004c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ac0 tx=0x7f2b28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e90d75ff416cff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:10.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.231+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c039070 con 0x7f2b28004800 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.231+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b2c02c6b0 con 0x7f2b28004800 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.231+0000 7f2b41600640 1 -- 172.21.15.32:0/544621695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c009690 con 0x7f2b28004800 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 -- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 msgr2=0x7f2b28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 0x7f2b28004c80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ac0 tx=0x7f2b28005830 comp rx=0 tx=0).stop 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 -- 172.21.15.32:0/544621695 shutdown_connections 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 --2- 172.21.15.32:0/544621695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b28004800 0x7f2b28004c80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 -- 172.21.15.32:0/544621695 >> 172.21.15.32:0/544621695 conn(0x7f2b4406cf00 msgr2=0x7f2b4406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:10.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 -- 172.21.15.32:0/544621695 shutdown_connections 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.232+0000 7f2b43e00640 1 -- 172.21.15.32:0/544621695 wait complete. 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.234+0000 7f2b43e00640 1 Processor -- start 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.234+0000 7f2b43e00640 1 -- start start 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b28004800 0x7f2b4415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 0x7f2b441f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b440cf880 con 0x7f2b441e82d0 2024-04-12T19:52:10.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b440b7b30 con 0x7f2b28004800 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b440d2e70 con 0x7f2b441f84a0 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.235+0000 7f2b42000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51452/0 (socket says 172.21.15.32:51452) 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 -- 172.21.15.32:0/3321842612 learned_addr learned my addr 172.21.15.32:0/3321842612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42a00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b28004800 0x7f2b4415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b43400640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 0x7f2b441f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 -- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 msgr2=0x7f2b441f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 0x7f2b441f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 -- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b28004800 msgr2=0x7f2b4415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b28004800 0x7f2b4415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b42000640 1 -- 172.21.15.32:0/3321842612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441581c0 con 0x7f2b441e82d0 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.236+0000 7f2b43400640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 0x7f2b441f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b42000640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ac0 tx=0x7f2b2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c039050 con 0x7f2b441e82d0 2024-04-12T19:52:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b2c045070 con 0x7f2b441e82d0 2024-04-12T19:52:10.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0044e0 con 0x7f2b441e82d0 2024-04-12T19:52:10.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4415ede0 con 0x7f2b441e82d0 2024-04-12T19:52:10.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.237+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4415d550 con 0x7f2b441e82d0 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.238+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b440d0b10 con 0x7f2b441e82d0 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.239+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b2c003a50 con 0x7f2b441e82d0 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.241+0000 7f2b3aa00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.241+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f2b2c0acf40 con 0x7f2b441e82d0 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.241+0000 7f2b42a00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.242+0000 7f2b42a00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 0x7f2b2006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b34003fd0 tx=0x7f2b34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.246+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b440d0b10 con 0x7f2b441e82d0 2024-04-12T19:52:10.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:10.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:10.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.453+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b440755c0 con 0x7f2b441e82d0 2024-04-12T19:52:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.455+0000 7f2b3aa00640 1 -- 172.21.15.32:0/3321842612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13008 (secure 0 0 0) 0x7f2b2c07b660 con 0x7f2b441e82d0 2024-04-12T19:52:10.456 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:10.456 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"fsid":"47426077-081d-47b8-bfde-ce036e9b4cf8","created":"2024-04-12T19:49:58.283528+0000","modified":"2024-04-12T19:52:04.811850+0000","last_up_change":"2024-04-12T19:50:11.423299+0000","last_in_change":"2024-04-12T19:50:02.461443+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T19:50:11.662919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7e3bc0f-a11b-4cd0-a6c5-f1db42ed9d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6817","nonce":737701280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6819","nonce":737701280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6828","nonce":737701280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":737701280},{"type":"v1","addr":"172.21.15.32:6823","nonce":737701280}]},"public_addr":"172.21.15.32:6817/737701280","cluster_addr":"172.21.15.32:6819/737701280","heartbeat_back_addr":"172.21.15.32:6828/737701280","heartbeat_front_addr":"172.21.15.32:6823/737701280","state":["exists","up"]},{"osd":1,"uuid":"083a68cd-ca10-4738-9980-41cf8ed276e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6801","nonce":4074798626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6803","nonce":4074798626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6807","nonce":4074798626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":4074798626},{"type":"v1","addr":"172.21.15.32:6805","nonce":4074798626}]},"public_addr":"172.21.15.32:6801/4074798626","cluster_addr":"172.21.15.32:6803/4074798626","heartbeat_back_addr":"172.21.15.32:6807/4074798626","heartbeat_front_addr":"172.21.15.32:6805/4074798626","state":["exists","up"]},{"osd":2,"uuid":"d78fb500-a0d8-453e-8930-c0a0dfaf6d26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6822","nonce":4204074222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6825","nonce":4204074222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6831","nonce":4204074222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":4204074222},{"type":"v1","addr":"172.21.15.32:6829","nonce":4204074222}]},"public_addr":"172.21.15.32:6822/4204074222","cluster_addr":"172.21.15.32:6825/4204074222","heartbeat_back_addr":"172.21.15.32:6831/4204074222","heartbeat_front_addr":"172.21.15.32:6829/4204074222","state":["exists","up"]},{"osd":3,"uuid":"e6e2c7ff-5821-4435-81bf-5d3265e5a5f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6809","nonce":3914231922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6811","nonce":3914231922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6815","nonce":3914231922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3914231922},{"type":"v1","addr":"172.21.15.32:6813","nonce":3914231922}]},"public_addr":"172.21.15.32:6809/3914231922","cluster_addr":"172.21.15.32:6811/3914231922","heartbeat_back_addr":"172.21.15.32:6815/3914231922","heartbeat_front_addr":"172.21.15.32:6813/3914231922","state":["exists","up"]},{"osd":4,"uuid":"b881395c-671e-4789-a57f-d80d3bdc3ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6801","nonce":3921333402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6803","nonce":3921333402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6807","nonce":3921333402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3921333402},{"type":"v1","addr":"172.21.15.169:6805","nonce":3921333402}]},"public_addr":"172.21.15.169:6801/3921333402","cluster_addr":"172.21.15.169:6803/3921333402","heartbeat_back_addr":"172.21.15.169:6807/3921333402","heartbeat_front_addr":"172.21.15.169:6805/3921333402","state":["exists","up"]},{"osd":5,"uuid":"f115ff76-a67b-4003-bd9f-758b919db1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6817","nonce":2535062120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6819","nonce":2535062120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6823","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6825","nonce":2535062120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2535062120},{"type":"v1","addr":"172.21.15.169:6821","nonce":2535062120}]},"public_addr":"172.21.15.169:6817/2535062120","cluster_addr":"172.21.15.169:6819/2535062120","heartbeat_back_addr":"172.21.15.169:6825/2535062120","heartbeat_front_addr":"172.21.15.169:6821/2535062120","state":["exists","up"]},{"osd":6,"uuid":"e4d48e5d-0727-437c-bdae-552adb15888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6809","nonce":2726062170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6811","nonce":2726062170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6815","nonce":2726062170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2726062170},{"type":"v1","addr":"172.21.15.169:6813","nonce":2726062170}]},"public_addr":"172.21.15.169:6809/2726062170","cluster_addr":"172.21.15.169:6811/2726062170","heartbeat_back_addr":"172.21.15.169:6815/2726062170","heartbeat_front_addr":"172.21.15.169:6813/2726062170","state":["exists","up"]},{"osd":7,"uuid":"7661f002-989b-4c53-8297-888c0c4529d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6824","nonce":883923800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6827","nonce":883923800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6831","nonce":883923800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":883923800},{"type":"v1","addr":"172.21.15.169:6829","nonce":883923800}]},"public_addr":"172.21.15.169:6824/883923800","cluster_addr":"172.21.15.169:6827/883923800","heartbeat_back_addr":"172.21.15.169:6831/883923800","heartbeat_front_addr":"172.21.15.169:6829/883923800","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.896412+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.848090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.914560+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.829723+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.765877+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.884588+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.858033+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T19:50:07.598713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/1336962272":"2024-04-13T19:52:01.717480+0000","172.21.15.32:6835/1336962272":"2024-04-13T19:52:01.717480+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T19:52:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.457+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 msgr2=0x7f2b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.457+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 0x7f2b2006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b34003fd0 tx=0x7f2b34009040 comp rx=0 tx=0).stop 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.457+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 msgr2=0x7f2b441e8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.457+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009ac0 tx=0x7f2b2c004730 comp rx=0 tx=0).stop 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 shutdown_connections 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b441f84a0 0x7f2b441f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b441e82d0 0x7f2b441e8750 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 --2- 172.21.15.32:0/3321842612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b28004800 0x7f2b4415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 >> 172.21.15.32:0/3321842612 conn(0x7f2b4406cf00 msgr2=0x7f2b440cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 shutdown_connections 2024-04-12T19:52:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:10.458+0000 7f2b43e00640 1 -- 172.21.15.32:0/3321842612 wait complete. 2024-04-12T19:52:11.477 INFO:tasks.ceph:Scrubbing osd.0 2024-04-12T19:52:11.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.711+0000 7fbfdaa00640 1 Processor -- start 2024-04-12T19:52:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.711+0000 7fbfdaa00640 1 -- start start 2024-04-12T19:52:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.713+0000 7fbfdaa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cec60 0x7fbfd4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.713+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.713+0000 7fbfdaa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd413fc70 con 0x7fbfd4072450 2024-04-12T19:52:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.713+0000 7fbfdaa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd40728c0 con 0x7fbfd40cec60 2024-04-12T19:52:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.713+0000 7fbfdaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd413f9e0 con 0x7fbfd415d420 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfd9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cec60 0x7fbfd4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfda000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfda000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.32:48414/0 (socket says 172.21.15.32:48414) 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfd9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfd40cec60 0x7fbfd4071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:35638/0 (socket says 172.21.15.32:35638) 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfda000640 1 -- 172.21.15.32:0/1172422489 learned_addr learned my addr 172.21.15.32:0/1172422489 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.714+0000 7fbfd8c00640 1 --1- 172.21.15.32:0/1172422489 >> v1:172.21.15.169:6789/0 conn(0x7fbfd4072450 0x7fbfd415cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:58266/0 (socket says 172.21.15.32:58266) 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfd9600640 1 -- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 msgr2=0x7fbfd406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfd9600640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfd9600640 1 -- 172.21.15.32:0/1172422489 >> v1:172.21.15.169:6789/0 conn(0x7fbfd4072450 legacy=0x7fbfd415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfd9600640 1 -- 172.21.15.32:0/1172422489 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd4141a00 con 0x7fbfd40cec60 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfda000640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:52:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.715+0000 7fbfd3e00640 1 -- 172.21.15.32:0/1172422489 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643099490 0 0) 0x7fbfd413fc70 con 0x7fbfd4072450 2024-04-12T19:52:11.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.716+0000 7fbfd9600640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4071f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80097d0 tx=0x7fbfc802b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e4f418d31ad581 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfd3e00640 1 -- 172.21.15.32:0/1172422489 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8035020 con 0x7fbfd40cec60 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfd3e00640 1 -- 172.21.15.32:0/1172422489 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc8031890 con 0x7fbfd40cec60 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfd3e00640 1 -- 172.21.15.32:0/1172422489 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8031c60 con 0x7fbfd40cec60 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 -- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 msgr2=0x7fbfd4071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4071f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80097d0 tx=0x7fbfc802b930 comp rx=0 tx=0).stop 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 -- 172.21.15.32:0/1172422489 shutdown_connections 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd406f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/1172422489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4071f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.717+0000 7fbfdaa00640 1 -- 172.21.15.32:0/1172422489 >> 172.21.15.32:0/1172422489 conn(0x7fbfd40b6170 msgr2=0x7fbfd406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.718+0000 7fbfdaa00640 1 -- 172.21.15.32:0/1172422489 shutdown_connections 2024-04-12T19:52:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.718+0000 7fbfdaa00640 1 -- 172.21.15.32:0/1172422489 wait complete. 2024-04-12T19:52:11.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.719+0000 7fbfdaa00640 1 Processor -- start 2024-04-12T19:52:11.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.720+0000 7fbfdaa00640 1 -- start start 2024-04-12T19:52:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.720+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 0x7fbfd4132950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd40b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfdaa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd41401f0 con 0x7fbfd4072450 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfdaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd413ff10 con 0x7fbfd40cec60 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfdaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd41404d0 con 0x7fbfd415d420 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 0x7fbfd4132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 0x7fbfd4132950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51462/0 (socket says 172.21.15.32:51462) 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 -- 172.21.15.32:0/2229812067 learned_addr learned my addr 172.21.15.32:0/2229812067 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:11.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 -- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 msgr2=0x7fbfd40b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfda000640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd40b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd8c00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd40b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 -- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 msgr2=0x7fbfd4158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40cec60 0x7fbfd4158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfd9600640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd4159800 con 0x7fbfd4072450 2024-04-12T19:52:11.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.721+0000 7fbfda000640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd415d420 0x7fbfd40b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:11.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfd9600640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 0x7fbfd4132950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8035050 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc803bcf0 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd4159ae0 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8004250 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.722+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd415a0a0 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.723+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbfd4068b40 con 0x7fbfd4072450 2024-04-12T19:52:11.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.724+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfc8038020 con 0x7fbfd4072450 2024-04-12T19:52:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.725+0000 7fbfd1600640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fbfb406c8f0 0x7fbfb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.725+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fbfc80ae2b0 con 0x7fbfd4072450 2024-04-12T19:52:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.725+0000 7fbfd1600640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7fbfb40727b0 0x7fbfb4074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.726+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfc800af30 con 0x7fbfb40727b0 2024-04-12T19:52:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.726+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbfc80ae660 con 0x7fbfd4072450 2024-04-12T19:52:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.726+0000 7fbfd8c00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fbfb406c8f0 0x7fbfb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.726+0000 7fbfda000640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7fbfb40727b0 0x7fbfb4074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.727+0000 7fbfd8c00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fbfb406c8f0 0x7fbfb406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007fd0 tx=0x7fbfc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.727+0000 7fbfda000640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7fbfb40727b0 0x7fbfb4074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.728+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== osd.0 v2:172.21.15.32:6816/737701280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbfc800af30 con 0x7fbfb40727b0 2024-04-12T19:52:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.742+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 --> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbfd40730e0 con 0x7fbfb40727b0 2024-04-12T19:52:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.743+0000 7fbfd1600640 1 -- 172.21.15.32:0/2229812067 <== osd.0 v2:172.21.15.32:6816/737701280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbfd40730e0 con 0x7fbfb40727b0 2024-04-12T19:52:11.743 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:11.744 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:11.744 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.743+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7fbfb40727b0 msgr2=0x7fbfb4074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.743+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6816/737701280,v1:172.21.15.32:6817/737701280] conn(0x7fbfb40727b0 0x7fbfb4074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fbfb406c8f0 msgr2=0x7fbfb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fbfb406c8f0 0x7fbfb406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007fd0 tx=0x7fbfc0009040 comp rx=0 tx=0).stop 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 msgr2=0x7fbfd4132950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 --2- 172.21.15.32:0/2229812067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfd4072450 0x7fbfd4132950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc80046c0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfda000640 1 -- 172.21.15.32:0/2229812067 reap_dead start 2024-04-12T19:52:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 shutdown_connections 2024-04-12T19:52:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 >> 172.21.15.32:0/2229812067 conn(0x7fbfd40b6170 msgr2=0x7fbfd40b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 shutdown_connections 2024-04-12T19:52:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.744+0000 7fbfdaa00640 1 -- 172.21.15.32:0/2229812067 wait complete. 2024-04-12T19:52:11.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-12T19:52:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.980+0000 7f1694a00640 1 Processor -- start 2024-04-12T19:52:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.980+0000 7f1694a00640 1 -- start start 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.981+0000 7f1694a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16900cf7f0 0x7f169006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.981+0000 7f1694a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 0x7f169006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.981+0000 7f1694a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16900d4650 con 0x7f16901620b0 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.981+0000 7f1694a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16900d2ea0 con 0x7f16900cf7f0 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.981+0000 7f1694a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16900d3180 con 0x7f16900cfc10 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16901620b0 0x7f16900d3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37400/0 (socket says 172.21.15.32:37400) 2024-04-12T19:52:11.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168e800640 1 -- 172.21.15.32:0/3133567892 learned_addr learned my addr 172.21.15.32:0/3133567892 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168de00640 1 --2- 172.21.15.32:0/3133567892 >> v2:172.21.15.32:3300/0 conn(0x7f16900cf7f0 0x7f169006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168f200640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 0x7f169006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168de00640 1 -- 172.21.15.32:0/3133567892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 msgr2=0x7f169006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.982+0000 7f168de00640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 0x7f169006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.983+0000 7f168de00640 1 -- 172.21.15.32:0/3133567892 >> v1:172.21.15.32:6789/0 conn(0x7f16901620b0 legacy=0x7f16900d3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.983+0000 7f168de00640 1 -- 172.21.15.32:0/3133567892 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16900d3460 con 0x7f16900cf7f0 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.983+0000 7f168f200640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 0x7f169006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:52:11.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.984+0000 7f168de00640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16900cf7f0 0x7f169006ce80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16780097a0 tx=0x7f167802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e68fd1f840721c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.984+0000 7f168d400640 1 -- 172.21.15.32:0/3133567892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678034070 con 0x7f16900cf7f0 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.984+0000 7f168d400640 1 -- 172.21.15.32:0/3133567892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1678030800 con 0x7f16900cf7f0 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.984+0000 7f168d400640 1 -- 172.21.15.32:0/3133567892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678030bd0 con 0x7f16900cf7f0 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 -- 172.21.15.32:0/3133567892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16900cf7f0 msgr2=0x7f169006ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16900cf7f0 0x7f169006ce80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16780097a0 tx=0x7f167802b940 comp rx=0 tx=0).stop 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 -- 172.21.15.32:0/3133567892 shutdown_connections 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cfc10 0x7f169006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 --2- 172.21.15.32:0/3133567892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16900cf7f0 0x7f169006ce80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 -- 172.21.15.32:0/3133567892 >> 172.21.15.32:0/3133567892 conn(0x7f16900d18d0 msgr2=0x7f16900d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 -- 172.21.15.32:0/3133567892 shutdown_connections 2024-04-12T19:52:11.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.985+0000 7f1694a00640 1 -- 172.21.15.32:0/3133567892 wait complete. 2024-04-12T19:52:11.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.987+0000 7f1694a00640 1 Processor -- start 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.987+0000 7f1694a00640 1 -- start start 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 0x7f1690150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16900cfc10 0x7f16901e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1690162490 con 0x7f16901620b0 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16900d3a20 con 0x7f16900cfc10 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f1694a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16900d5550 con 0x7f16900cf7f0 2024-04-12T19:52:11.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51482/0 (socket says 172.21.15.32:51482) 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168f200640 1 -- 172.21.15.32:0/2932246831 learned_addr learned my addr 172.21.15.32:0/2932246831 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168de00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16900cfc10 0x7f16901e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168e800640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 0x7f1690150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.988+0000 7f168f200640 1 -- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 msgr2=0x7f1690150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168f200640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 0x7f1690150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168f200640 1 -- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16900cfc10 msgr2=0x7f16901e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168f200640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16900cfc10 0x7f16901e7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:11.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168f200640 1 -- 172.21.15.32:0/2932246831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16900c9170 con 0x7f16901620b0 2024-04-12T19:52:11.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168e800640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 0x7f1690150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f168f200640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1688007f20 tx=0x7f168800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168801a070 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1688003b90 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688004030 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16900c9450 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.989+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16900c9a10 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.990+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16900755c0 con 0x7f16901620b0 2024-04-12T19:52:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.991+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f168801d070 con 0x7f16901620b0 2024-04-12T19:52:11.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.992+0000 7f1686a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 0x7f166c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:11.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.992+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f168808bb70 con 0x7f16901620b0 2024-04-12T19:52:11.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.993+0000 7f168e800640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 0x7f166c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.993+0000 7f168e800640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 0x7f166c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1680000f60 tx=0x7f1680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:11.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:11.997+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1688010070 con 0x7f16901620b0 2024-04-12T19:52:12.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.201+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f16900cf1a0 con 0x7f166c06c8f0 2024-04-12T19:52:12.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.202+0000 7f1686a00640 1 -- 172.21.15.32:0/2932246831 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f16900cf1a0 con 0x7f166c06c8f0 2024-04-12T19:52:12.203 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-04-12T19:52:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 msgr2=0x7f166c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 0x7f166c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1680000f60 tx=0x7f1680009040 comp rx=0 tx=0).stop 2024-04-12T19:52:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 msgr2=0x7f16900ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1688007f20 tx=0x7f168800d560 comp rx=0 tx=0).stop 2024-04-12T19:52:12.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 shutdown_connections 2024-04-12T19:52:12.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f166c06c8f0 0x7f166c06edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16901620b0 0x7f16900ccee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16900cfc10 0x7f16901e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 --2- 172.21.15.32:0/2932246831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16900cf7f0 0x7f1690150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 >> 172.21.15.32:0/2932246831 conn(0x7f16900d18d0 msgr2=0x7f1690160d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:12.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 shutdown_connections 2024-04-12T19:52:12.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.205+0000 7f1694a00640 1 -- 172.21.15.32:0/2932246831 wait complete. 2024-04-12T19:52:12.225 INFO:tasks.ceph:Scrubbing osd.1 2024-04-12T19:52:12.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.456+0000 7fb94b200640 1 Processor -- start 2024-04-12T19:52:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.457+0000 7fb94b200640 1 -- start start 2024-04-12T19:52:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.458+0000 7fb94b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 0x7fb9440cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.458+0000 7fb94b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9440b78e0 0x7fb9440b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.458+0000 7fb94b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9440cfe30 con 0x7fb9440cf510 2024-04-12T19:52:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.458+0000 7fb94b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb9440d3f60 con 0x7fb9440b78e0 2024-04-12T19:52:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.458+0000 7fb94b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9440d3350 con 0x7fb9440b6b80 2024-04-12T19:52:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9440b78e0 0x7fb9440b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9440b78e0 0x7fb9440b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51488/0 (socket says 172.21.15.32:51488) 2024-04-12T19:52:12.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 -- 172.21.15.32:0/863967013 learned_addr learned my addr 172.21.15.32:0/863967013 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:12.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb949400640 1 --2- 172.21.15.32:0/863967013 >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 0x7fb9440cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 -- 172.21.15.32:0/863967013 >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 msgr2=0x7fb9440cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:12.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 --2- 172.21.15.32:0/863967013 >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 0x7fb9440cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 -- 172.21.15.32:0/863967013 >> v1:172.21.15.169:6789/0 conn(0x7fb9440cf510 legacy=0x7fb9440d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb94a800640 1 -- 172.21.15.32:0/863967013 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9440d3630 con 0x7fb9440b78e0 2024-04-12T19:52:12.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.459+0000 7fb949400640 1 --2- 172.21.15.32:0/863967013 >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 0x7fb9440cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:52:12.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.461+0000 7fb94a800640 1 --2- 172.21.15.32:0/863967013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b78e0 0x7fb9440b7d70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9440cb7c0 tx=0x7fb94002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f88c9719c9ff2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.461+0000 7fb948a00640 1 -- 172.21.15.32:0/863967013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940034070 con 0x7fb9440b78e0 2024-04-12T19:52:12.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.461+0000 7fb948a00640 1 -- 172.21.15.32:0/863967013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940030860 con 0x7fb9440b78e0 2024-04-12T19:52:12.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.461+0000 7fb948a00640 1 -- 172.21.15.32:0/863967013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940030c30 con 0x7fb9440b78e0 2024-04-12T19:52:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 -- 172.21.15.32:0/863967013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b78e0 msgr2=0x7fb9440b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 --2- 172.21.15.32:0/863967013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b78e0 0x7fb9440b7d70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9440cb7c0 tx=0x7fb94002b9a0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 -- 172.21.15.32:0/863967013 shutdown_connections 2024-04-12T19:52:12.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 --2- 172.21.15.32:0/863967013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b78e0 0x7fb9440b7d70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 --2- 172.21.15.32:0/863967013 >> v2:172.21.15.169:3300/0 conn(0x7fb9440b6b80 0x7fb9440cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 -- 172.21.15.32:0/863967013 >> 172.21.15.32:0/863967013 conn(0x7fb9440d13a0 msgr2=0x7fb9440d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 -- 172.21.15.32:0/863967013 shutdown_connections 2024-04-12T19:52:12.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.462+0000 7fb94b200640 1 -- 172.21.15.32:0/863967013 wait complete. 2024-04-12T19:52:12.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.464+0000 7fb94b200640 1 Processor -- start 2024-04-12T19:52:12.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.464+0000 7fb94b200640 1 -- start start 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 0x7fb9440cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b78e0 0x7fb9440c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9440cf510 0x7fb9440cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb9440d43f0 con 0x7fb9440b6b80 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9440d3bf0 con 0x7fb9440cf510 2024-04-12T19:52:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.465+0000 7fb94b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9440d4570 con 0x7fb9440b78e0 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 0x7fb9440cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 0x7fb9440cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51500/0 (socket says 172.21.15.32:51500) 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 -- 172.21.15.32:0/585215862 learned_addr learned my addr 172.21.15.32:0/585215862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb94a800640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9440cf510 0x7fb9440cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949400640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b78e0 0x7fb9440c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 -- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b78e0 msgr2=0x7fb9440c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b78e0 0x7fb9440c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 -- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9440cf510 msgr2=0x7fb9440cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9440cf510 0x7fb9440cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb944156d00 con 0x7fb9440b6b80 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.466+0000 7fb949e00640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 0x7fb9440cad70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c870 tx=0x7fb93c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.467+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c003b70 con 0x7fb9440b6b80 2024-04-12T19:52:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.467+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb93c003d30 con 0x7fb9440b6b80 2024-04-12T19:52:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.467+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00aa90 con 0x7fb9440b6b80 2024-04-12T19:52:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.467+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb944156fe0 con 0x7fb9440b6b80 2024-04-12T19:52:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.467+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441575a0 con 0x7fb9440b6b80 2024-04-12T19:52:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.468+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9440cd5a0 con 0x7fb9440b6b80 2024-04-12T19:52:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.469+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb93c00ac30 con 0x7fb9440b6b80 2024-04-12T19:52:12.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb93a000640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fb93c08c270 con 0x7fb9440b6b80 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb93a000640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fb9280727b0 0x7fb928074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb949400640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb928075280 con 0x7fb9280727b0 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.470+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb93c08c620 con 0x7fb9440b6b80 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.471+0000 7fb94a800640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fb9280727b0 0x7fb928074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.471+0000 7fb949400640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb9440c9fc0 tx=0x7fb934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.471+0000 7fb94a800640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fb9280727b0 0x7fb928074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.472+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== osd.1 v2:172.21.15.32:6800/4074798626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb928075280 con 0x7fb9280727b0 2024-04-12T19:52:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.487+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 --> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb9440b7430 con 0x7fb9280727b0 2024-04-12T19:52:12.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.487+0000 7fb93a000640 1 -- 172.21.15.32:0/585215862 <== osd.1 v2:172.21.15.32:6800/4074798626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb9440b7430 con 0x7fb9280727b0 2024-04-12T19:52:12.488 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:12.489 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:12.489 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:12.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fb9280727b0 msgr2=0x7fb928074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6800/4074798626,v1:172.21.15.32:6801/4074798626] conn(0x7fb9280727b0 0x7fb928074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb92806c8f0 msgr2=0x7fb92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb9440c9fc0 tx=0x7fb934008040 comp rx=0 tx=0).stop 2024-04-12T19:52:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 msgr2=0x7fb9440cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 --2- 172.21.15.32:0/585215862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9440b6b80 0x7fb9440cad70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c870 tx=0x7fb93c00cd30 comp rx=0 tx=0).stop 2024-04-12T19:52:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94a800640 1 -- 172.21.15.32:0/585215862 reap_dead start 2024-04-12T19:52:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 shutdown_connections 2024-04-12T19:52:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.488+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 >> 172.21.15.32:0/585215862 conn(0x7fb9440d13a0 msgr2=0x7fb9440b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.489+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 shutdown_connections 2024-04-12T19:52:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.489+0000 7fb94b200640 1 -- 172.21.15.32:0/585215862 wait complete. 2024-04-12T19:52:12.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-12T19:52:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.717+0000 7f6022600640 1 Processor -- start 2024-04-12T19:52:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.717+0000 7f6022600640 1 -- start start 2024-04-12T19:52:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6022600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f601c0d1550 0x7f601c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6022600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c0d32e0 0x7f601c0d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6022600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601c0b78f0 con 0x7f601c15eab0 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6022600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f601c0d1ef0 con 0x7f601c0d1550 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6022600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f601c0d2e50 con 0x7f601c0d32e0 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6021200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f601c15eab0 0x7f601c162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37416/0 (socket says 172.21.15.32:37416) 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.719+0000 7f6021200640 1 -- 172.21.15.32:0/3885481285 learned_addr learned my addr 172.21.15.32:0/3885481285 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601be00640 1 --2- 172.21.15.32:0/3885481285 >> v2:172.21.15.32:3300/0 conn(0x7f601c0d1550 0x7f601c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f6021c00640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c0d32e0 0x7f601c0d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807494339 0 0) 0x7f6010001110 con 0x7f601c15eab0 2024-04-12T19:52:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6004003520 con 0x7f601c15eab0 2024-04-12T19:52:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601be00640 1 -- 172.21.15.32:0/3885481285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c0d32e0 msgr2=0x7f601c0d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601be00640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c0d32e0 0x7f601c0d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601be00640 1 -- 172.21.15.32:0/3885481285 >> v1:172.21.15.32:6789/0 conn(0x7f601c15eab0 legacy=0x7f601c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132792388 0 0) 0x7f6004003520 con 0x7f601c15eab0 2024-04-12T19:52:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.720+0000 7f601be00640 1 -- 172.21.15.32:0/3885481285 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c162d40 con 0x7f601c0d1550 2024-04-12T19:52:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.721+0000 7f601be00640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0d19b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60080097a0 tx=0x7f600802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1642429b573d2bef server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008037070 con 0x7f601c0d1550 2024-04-12T19:52:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6008030510 con 0x7f601c0d1550 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f601b400640 1 -- 172.21.15.32:0/3885481285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010001840 con 0x7f601c0d1550 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 -- 172.21.15.32:0/3885481285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 msgr2=0x7f601c0d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0d19b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f60080097a0 tx=0x7f600802b790 comp rx=0 tx=0).stop 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 -- 172.21.15.32:0/3885481285 shutdown_connections 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c0d32e0 0x7f601c0d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 --2- 172.21.15.32:0/3885481285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0d19b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 -- 172.21.15.32:0/3885481285 >> 172.21.15.32:0/3885481285 conn(0x7f601c073240 msgr2=0x7f601c073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.722+0000 7f6022600640 1 -- 172.21.15.32:0/3885481285 shutdown_connections 2024-04-12T19:52:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.723+0000 7f6022600640 1 -- 172.21.15.32:0/3885481285 wait complete. 2024-04-12T19:52:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.724+0000 7f6022600640 1 Processor -- start 2024-04-12T19:52:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.724+0000 7f6022600640 1 -- start start 2024-04-12T19:52:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601c0d32e0 0x7f601c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c15eab0 0x7f601c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0d2ae0 con 0x7f601c0d1550 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1632f0 con 0x7f601c0d32e0 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.725+0000 7f6022600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f601c15ee90 con 0x7f601c15eab0 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51524/0 (socket says 172.21.15.32:51524) 2024-04-12T19:52:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 -- 172.21.15.32:0/3495746105 learned_addr learned my addr 172.21.15.32:0/3495746105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f601be00640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601c0d32e0 0x7f601c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 -- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c15eab0 msgr2=0x7f601c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c15eab0 0x7f601c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 -- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601c0d32e0 msgr2=0x7f601c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601c0d32e0 0x7f601c0cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 -- 172.21.15.32:0/3495746105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c15ac50 con 0x7f601c0d1550 2024-04-12T19:52:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.726+0000 7f6021200640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f601000c750 tx=0x7f601000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.727+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010005ad0 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.727+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6010005c90 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.727+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100072b0 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.727+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c15af30 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.727+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c15b4f0 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.728+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c0755c0 con 0x7f601c0d1550 2024-04-12T19:52:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.729+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6010007450 con 0x7f601c0d1550 2024-04-12T19:52:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.730+0000 7f6018c00640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 0x7f5ffc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.730+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f601008b760 con 0x7f601c0d1550 2024-04-12T19:52:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.730+0000 7f601be00640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 0x7f5ffc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:12.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.731+0000 7f601be00640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 0x7f5ffc06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f600802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.735+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f601c0b78f0 con 0x7f601c0d1550 2024-04-12T19:52:12.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.938+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f601c0d2460 con 0x7f5ffc06c8f0 2024-04-12T19:52:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.939+0000 7f6018c00640 1 -- 172.21.15.32:0/3495746105 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f601c0d2460 con 0x7f5ffc06c8f0 2024-04-12T19:52:12.939 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-04-12T19:52:12.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 msgr2=0x7f5ffc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 0x7f5ffc06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f600802f000 comp rx=0 tx=0).stop 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 msgr2=0x7f601c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f601000c750 tx=0x7f601000cc10 comp rx=0 tx=0).stop 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 shutdown_connections 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ffc06c8f0 0x7f5ffc06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f601c15eab0 0x7f601c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601c0d32e0 0x7f601c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 --2- 172.21.15.32:0/3495746105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c0d1550 0x7f601c0cd8a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 >> 172.21.15.32:0/3495746105 conn(0x7f601c073240 msgr2=0x7f601c15f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 shutdown_connections 2024-04-12T19:52:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:12.941+0000 7f6022600640 1 -- 172.21.15.32:0/3495746105 wait complete. 2024-04-12T19:52:12.961 INFO:tasks.ceph:Scrubbing osd.2 2024-04-12T19:52:12.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.186+0000 7fdde8c00640 1 Processor -- start 2024-04-12T19:52:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.187+0000 7fdde8c00640 1 -- start start 2024-04-12T19:52:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.188+0000 7fdde8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdde40cec60 0x7fdde4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.188+0000 7fdde8c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdde4072450 0x7fdde415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.188+0000 7fdde8c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde413fc70 con 0x7fdde415d250 2024-04-12T19:52:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.188+0000 7fdde8c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdde40728c0 con 0x7fdde4072450 2024-04-12T19:52:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.188+0000 7fdde8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdde413f9e0 con 0x7fdde40cec60 2024-04-12T19:52:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdde3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdde415d250 0x7fdde406f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37430/0 (socket says 172.21.15.32:37430) 2024-04-12T19:52:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdddbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdde4072450 0x7fdde415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdde3e00640 1 -- 172.21.15.32:0/3509433471 learned_addr learned my addr 172.21.15.32:0/3509433471 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdde3400640 1 --2- 172.21.15.32:0/3509433471 >> v2:172.21.15.169:3300/0 conn(0x7fdde40cec60 0x7fdde4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdde2a00640 1 -- 172.21.15.32:0/3509433471 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430121939 0 0) 0x7fdde413fc70 con 0x7fdde415d250 2024-04-12T19:52:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdde2a00640 1 -- 172.21.15.32:0/3509433471 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc8003520 con 0x7fdde415d250 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdddbe00640 1 -- 172.21.15.32:0/3509433471 >> v2:172.21.15.169:3300/0 conn(0x7fdde40cec60 msgr2=0x7fdde4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.189+0000 7fdddbe00640 1 --2- 172.21.15.32:0/3509433471 >> v2:172.21.15.169:3300/0 conn(0x7fdde40cec60 0x7fdde4071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.190+0000 7fdddbe00640 1 -- 172.21.15.32:0/3509433471 >> v1:172.21.15.32:6789/0 conn(0x7fdde415d250 legacy=0x7fdde406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.190+0000 7fdddbe00640 1 -- 172.21.15.32:0/3509433471 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde4141a00 con 0x7fdde4072450 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.191+0000 7fdddbe00640 1 --2- 172.21.15.32:0/3509433471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde4072450 0x7fdde415cd10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097a0 tx=0x7fddcc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc9cb52df6915a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.191+0000 7fdde2a00640 1 -- 172.21.15.32:0/3509433471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc034070 con 0x7fdde4072450 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.191+0000 7fdde2a00640 1 -- 172.21.15.32:0/3509433471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc030610 con 0x7fdde4072450 2024-04-12T19:52:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.191+0000 7fdde2a00640 1 -- 172.21.15.32:0/3509433471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0309e0 con 0x7fdde4072450 2024-04-12T19:52:13.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 -- 172.21.15.32:0/3509433471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde4072450 msgr2=0x7fdde415cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 --2- 172.21.15.32:0/3509433471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde4072450 0x7fdde415cd10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097a0 tx=0x7fddcc02b960 comp rx=0 tx=0).stop 2024-04-12T19:52:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 -- 172.21.15.32:0/3509433471 shutdown_connections 2024-04-12T19:52:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 --2- 172.21.15.32:0/3509433471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde4072450 0x7fdde415cd10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 --2- 172.21.15.32:0/3509433471 >> v2:172.21.15.169:3300/0 conn(0x7fdde40cec60 0x7fdde4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 -- 172.21.15.32:0/3509433471 >> 172.21.15.32:0/3509433471 conn(0x7fdde40b6170 msgr2=0x7fdde406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 -- 172.21.15.32:0/3509433471 shutdown_connections 2024-04-12T19:52:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.192+0000 7fdde8c00640 1 -- 172.21.15.32:0/3509433471 wait complete. 2024-04-12T19:52:13.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.194+0000 7fdde8c00640 1 Processor -- start 2024-04-12T19:52:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.194+0000 7fdde8c00640 1 -- start start 2024-04-12T19:52:13.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde4072450 0x7fdde413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde40cec60 0x7fdde4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 0x7fdde40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdde41401f0 con 0x7fdde415d250 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdde413ff10 con 0x7fdde4072450 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdde41404d0 con 0x7fdde40cec60 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 0x7fdde40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 0x7fdde40b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51536/0 (socket says 172.21.15.32:51536) 2024-04-12T19:52:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde3e00640 1 -- 172.21.15.32:0/2240591818 learned_addr learned my addr 172.21.15.32:0/2240591818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdde3400640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde4072450 0x7fdde413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.195+0000 7fdddbe00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde40cec60 0x7fdde4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 -- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde40cec60 msgr2=0x7fdde4157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde40cec60 0x7fdde4157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 -- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde4072450 msgr2=0x7fdde413c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde4072450 0x7fdde413c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde4153f30 con 0x7fdde415d250 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdddbe00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde40cec60 0x7fdde4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde3e00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 0x7fdde40b4e60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00caa0 tx=0x7fdddc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc017070 con 0x7fdde415d250 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc00ac00 con 0x7fdde415d250 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc012400 con 0x7fdde415d250 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde4154210 con 0x7fdde415d250 2024-04-12T19:52:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.196+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41547d0 con 0x7fdde415d250 2024-04-12T19:52:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.197+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdde4153ce0 con 0x7fdde415d250 2024-04-12T19:52:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.198+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdddc01a070 con 0x7fdde415d250 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.199+0000 7fdddb400640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fdddc08b680 con 0x7fdde415d250 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdddb400640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7fddc00727b0 0x7fddc0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddc0075280 con 0x7fddc00727b0 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdde3400640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdddc092050 con 0x7fdde415d250 2024-04-12T19:52:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.200+0000 7fdddbe00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7fddc00727b0 0x7fddc0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.201+0000 7fdde3400640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009ad0 tx=0x7fddd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.201+0000 7fdddbe00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7fddc00727b0 0x7fddc0074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.202+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== osd.2 v2:172.21.15.32:6820/4204074222 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddc0075280 con 0x7fddc00727b0 2024-04-12T19:52:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.216+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 --> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdde4068b40 con 0x7fddc00727b0 2024-04-12T19:52:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdddb400640 1 -- 172.21.15.32:0/2240591818 <== osd.2 v2:172.21.15.32:6820/4204074222 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdde4068b40 con 0x7fddc00727b0 2024-04-12T19:52:13.217 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:13.217 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:13.218 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7fddc00727b0 msgr2=0x7fddc0074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6820/4204074222,v1:172.21.15.32:6822/4204074222] conn(0x7fddc00727b0 0x7fddc0074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddc006c8f0 msgr2=0x7fddc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009ad0 tx=0x7fddd4008040 comp rx=0 tx=0).stop 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 msgr2=0x7fdde40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 --2- 172.21.15.32:0/2240591818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdde415d250 0x7fdde40b4e60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00caa0 tx=0x7fdddc00cf60 comp rx=0 tx=0).stop 2024-04-12T19:52:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde3e00640 1 -- 172.21.15.32:0/2240591818 reap_dead start 2024-04-12T19:52:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 shutdown_connections 2024-04-12T19:52:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 >> 172.21.15.32:0/2240591818 conn(0x7fdde40b6170 msgr2=0x7fdde415b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.217+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 shutdown_connections 2024-04-12T19:52:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.218+0000 7fdde8c00640 1 -- 172.21.15.32:0/2240591818 wait complete. 2024-04-12T19:52:13.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-12T19:52:13.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.455+0000 7fac86400640 1 Processor -- start 2024-04-12T19:52:13.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.455+0000 7fac86400640 1 -- start start 2024-04-12T19:52:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac86400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac80148050 0x7fac800b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac86400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac800b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac86400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac800afba0 con 0x7fac800ae790 2024-04-12T19:52:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac86400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fac800aed00 con 0x7fac80148050 2024-04-12T19:52:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac86400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac800aefe0 con 0x7fac80162b30 2024-04-12T19:52:13.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac7fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac800ae790 0x7fac80162420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37436/0 (socket says 172.21.15.32:37436) 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.457+0000 7fac7fe00640 1 -- 172.21.15.32:0/3899979215 learned_addr learned my addr 172.21.15.32:0/3899979215 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac85000640 1 --2- 172.21.15.32:0/3899979215 >> v2:172.21.15.169:3300/0 conn(0x7fac80148050 0x7fac800b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac85a00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac800b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2957246398 0 0) 0x7fac800afba0 con 0x7fac800ae790 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac64003520 con 0x7fac800ae790 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798214911 0 0) 0x7fac64003520 con 0x7fac800ae790 2024-04-12T19:52:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 msgr2=0x7fac800b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac800b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 >> v2:172.21.15.169:3300/0 conn(0x7fac80148050 msgr2=0x7fac800b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 --2- 172.21.15.32:0/3899979215 >> v2:172.21.15.169:3300/0 conn(0x7fac80148050 0x7fac800b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800af2c0 con 0x7fac800ae790 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fac6c001670 con 0x7fac800ae790 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 >> v1:172.21.15.32:6789/0 conn(0x7fac800ae790 legacy=0x7fac80162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640039b0 0x7fac64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.458+0000 7fac7f400640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7f400640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac64004d40 0x7fac640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fac800aed00 con 0x7fac64004d40 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac800aefe0 con 0x7fac640039b0 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac6400b9c0 con 0x7fac80162b30 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac64004d40 0x7fac640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85000640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640039b0 0x7fac64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7fe00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 -- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 msgr2=0x7fac64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 -- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640039b0 msgr2=0x7fac64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640039b0 0x7fac64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac85a00640 1 -- 172.21.15.32:0/3899979215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac64003520 con 0x7fac64004d40 2024-04-12T19:52:13.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.459+0000 7fac7fe00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac80162b30 0x7fac64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac85a00640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac64004d40 0x7fac640051c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fc0 tx=0x7fac64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163a8aa6e1dfe1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78039070 con 0x7fac64004d40 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7802cc00 con 0x7fac64004d40 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac7f400640 1 -- 172.21.15.32:0/3899979215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78009c10 con 0x7fac64004d40 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac86400640 1 -- 172.21.15.32:0/3899979215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac64004d40 msgr2=0x7fac640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac86400640 1 --2- 172.21.15.32:0/3899979215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac64004d40 0x7fac640051c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fc0 tx=0x7fac64005830 comp rx=0 tx=0).stop 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac85a00640 1 -- 172.21.15.32:0/3899979215 reap_dead start 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac86400640 1 -- 172.21.15.32:0/3899979215 shutdown_connections 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.461+0000 7fac86400640 1 -- 172.21.15.32:0/3899979215 >> 172.21.15.32:0/3899979215 conn(0x7fac800b23a0 msgr2=0x7fac800b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.462+0000 7fac86400640 1 -- 172.21.15.32:0/3899979215 shutdown_connections 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.462+0000 7fac86400640 1 -- 172.21.15.32:0/3899979215 wait complete. 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.463+0000 7fac86400640 1 Processor -- start 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 -- start start 2024-04-12T19:52:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 0x7fac80141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8013ce50 0x7fac8013d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fac801485b0 con 0x7fac801f7920 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac80148430 con 0x7fac64004d40 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.464+0000 7fac86400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac800b3cc0 con 0x7fac8013ce50 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51550/0 (socket says 172.21.15.32:51550) 2024-04-12T19:52:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 -- 172.21.15.32:0/2655887299 learned_addr learned my addr 172.21.15.32:0/2655887299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac85000640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 0x7fac80141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac85a00640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8013ce50 0x7fac8013d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 -- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8013ce50 msgr2=0x7fac8013d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8013ce50 0x7fac8013d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 -- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 msgr2=0x7fac80141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 0x7fac80141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac7fe00640 1 -- 172.21.15.32:0/2655887299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8013d7f0 con 0x7fac801f7920 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.465+0000 7fac85000640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 0x7fac80141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac7fe00640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac6c001200 tx=0x7fac6c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c019070 con 0x7fac801f7920 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c0074e0 con 0x7fac801f7920 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801f8420 con 0x7fac801f7920 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c007690 con 0x7fac801f7920 2024-04-12T19:52:13.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.466+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac800ce450 con 0x7fac801f7920 2024-04-12T19:52:13.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.467+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac800755c0 con 0x7fac801f7920 2024-04-12T19:52:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.468+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac6c01c070 con 0x7fac801f7920 2024-04-12T19:52:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.469+0000 7fac7cc00640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.469+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fac6c08b480 con 0x7fac801f7920 2024-04-12T19:52:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.469+0000 7fac85000640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.470+0000 7fac85000640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 0x7fac6006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac70009a80 tx=0x7fac70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.474+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac6c059b10 con 0x7fac801f7920 2024-04-12T19:52:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.685+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fac800b3240 con 0x7fac6006c8f0 2024-04-12T19:52:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.685+0000 7fac7cc00640 1 -- 172.21.15.32:0/2655887299 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fac800b3240 con 0x7fac6006c8f0 2024-04-12T19:52:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-04-12T19:52:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 msgr2=0x7fac6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 0x7fac6006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac70009a80 tx=0x7fac70008040 comp rx=0 tx=0).stop 2024-04-12T19:52:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 msgr2=0x7fac801f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac6c001200 tx=0x7fac6c00dd20 comp rx=0 tx=0).stop 2024-04-12T19:52:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 shutdown_connections 2024-04-12T19:52:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8013ce50 0x7fac8013d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac801f7920 0x7fac801f7da0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 --2- 172.21.15.32:0/2655887299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64004d40 0x7fac80141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 >> 172.21.15.32:0/2655887299 conn(0x7fac800b23a0 msgr2=0x7fac8006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 shutdown_connections 2024-04-12T19:52:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.688+0000 7fac86400640 1 -- 172.21.15.32:0/2655887299 wait complete. 2024-04-12T19:52:13.708 INFO:tasks.ceph:Scrubbing osd.3 2024-04-12T19:52:13.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.938+0000 7f728be00640 1 Processor -- start 2024-04-12T19:52:13.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.938+0000 7f728be00640 1 -- start start 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.939+0000 7f728be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f728c0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c0cfe30 con 0x7f728c0cf510 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c0d3350 con 0x7f728c0b78e0 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0d3f60 con 0x7f728c0b6b80 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f728c0cf510 0x7f728c0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50386/0 (socket says 172.21.15.32:50386) 2024-04-12T19:52:13.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728aa00640 1 -- 172.21.15.32:0/3058671061 learned_addr learned my addr 172.21.15.32:0/3058671061 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756258533 0 0) 0x7f728c0cfe30 con 0x7f728c0cf510 2024-04-12T19:52:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f726c003520 con 0x7f728c0cf510 2024-04-12T19:52:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.940+0000 7f7282000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f728c0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2799377457 0 0) 0x7f726c003520 con 0x7f728c0cf510 2024-04-12T19:52:13.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 msgr2=0x7f728c0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f728c0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 >> v1:172.21.15.169:6789/0 conn(0x7f728c0b78e0 legacy=0x7f728c0b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0d3630 con 0x7f728c0cf510 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f727c001670 con 0x7f728c0cf510 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 >> v1:172.21.15.32:6789/0 conn(0x7f728c0cf510 legacy=0x7f728c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726c0039e0 0x7f726c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f726c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c004d40 0x7f726c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0d3f60 con 0x7f726c004d40 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f726c00bb50 con 0x7f726c0039e0 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f726c00bd20 con 0x7f728c0b6b80 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.941+0000 7f728b400640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c004d40 0x7f726c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728aa00640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726c0039e0 0x7f726c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f7282000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f726c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728b400640 1 -- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 msgr2=0x7f726c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728b400640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f726c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728b400640 1 -- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726c0039e0 msgr2=0x7f726c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728b400640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726c0039e0 0x7f726c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f728b400640 1 -- 172.21.15.32:0/3058671061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c003520 con 0x7f726c004d40 2024-04-12T19:52:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.942+0000 7f7282000640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f728c0b6b80 0x7f726c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.943+0000 7f728b400640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c004d40 0x7f726c0051c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7284001e80 tx=0x7f726c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5011c34cbc13f257 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.943+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284001a70 con 0x7f726c004d40 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.943+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7284003b70 con 0x7f726c004d40 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.943+0000 7f728a000640 1 -- 172.21.15.32:0/3058671061 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72840050d0 con 0x7f726c004d40 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 -- 172.21.15.32:0/3058671061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c004d40 msgr2=0x7f726c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 --2- 172.21.15.32:0/3058671061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c004d40 0x7f726c0051c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7284001e80 tx=0x7f726c005830 comp rx=0 tx=0).stop 2024-04-12T19:52:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728b400640 1 -- 172.21.15.32:0/3058671061 reap_dead start 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 -- 172.21.15.32:0/3058671061 shutdown_connections 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 -- 172.21.15.32:0/3058671061 >> 172.21.15.32:0/3058671061 conn(0x7f728c0d13a0 msgr2=0x7f728c0d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 -- 172.21.15.32:0/3058671061 shutdown_connections 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.944+0000 7f728be00640 1 -- 172.21.15.32:0/3058671061 wait complete. 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.946+0000 7f728be00640 1 Processor -- start 2024-04-12T19:52:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.946+0000 7f728be00640 1 -- start start 2024-04-12T19:52:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c004d40 0x7f728c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728c0c8890 0x7f728c1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 0x7f728c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0d43f0 con 0x7f728c1f4c40 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0d3bf0 con 0x7f728c0c8890 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0d4570 con 0x7f726c004d40 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 0x7f728c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 0x7f728c1f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48638/0 (socket says 172.21.15.32:48638) 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.947+0000 7f728b400640 1 -- 172.21.15.32:0/528907774 learned_addr learned my addr 172.21.15.32:0/528907774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728aa00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c004d40 0x7f728c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f7282000640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728c0c8890 0x7f728c1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 -- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c004d40 msgr2=0x7f728c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c004d40 0x7f728c0c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 -- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728c0c8890 msgr2=0x7f728c1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f728c0c8890 0x7f728c1f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0c9080 con 0x7f728c1f4c40 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728aa00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c004d40 0x7f728c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728b400640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 0x7f728c1f50f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7284005040 tx=0x7f72840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284007db0 con 0x7f728c1f4c40 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7284003970 con 0x7f728c1f4c40 2024-04-12T19:52:13.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284006d70 con 0x7f728c1f4c40 2024-04-12T19:52:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.948+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c0c9360 con 0x7f728c1f4c40 2024-04-12T19:52:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.949+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c0c9920 con 0x7f728c1f4c40 2024-04-12T19:52:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.950+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f728c068b40 con 0x7f728c1f4c40 2024-04-12T19:52:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.951+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7284041030 con 0x7f728c1f4c40 2024-04-12T19:52:13.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7283400640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f726806c920 0x7f726806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f72840ac870 con 0x7f728c1f4c40 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7283400640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7f72680727e0 0x7f7268074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7284009eb0 con 0x7f72680727e0 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f728aa00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f726806c920 0x7f726806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f72840b12a0 con 0x7f728c1f4c40 2024-04-12T19:52:13.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.952+0000 7f7282000640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7f72680727e0 0x7f7268074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.953+0000 7f728aa00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f726806c920 0x7f726806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f727c001eb0 tx=0x7f727c001da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.953+0000 7f7282000640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7f72680727e0 0x7f7268074c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:13.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.954+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== osd.3 v2:172.21.15.32:6808/3914231922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7284009eb0 con 0x7f72680727e0 2024-04-12T19:52:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.969+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 --> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f728c0b8b70 con 0x7f72680727e0 2024-04-12T19:52:13.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f7283400640 1 -- 172.21.15.32:0/528907774 <== osd.3 v2:172.21.15.32:6808/3914231922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f728c0b8b70 con 0x7f72680727e0 2024-04-12T19:52:13.970 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:13.970 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:13.970 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7f72680727e0 msgr2=0x7f7268074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6808/3914231922,v1:172.21.15.32:6809/3914231922] conn(0x7f72680727e0 0x7f7268074c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f726806c920 msgr2=0x7f726806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f726806c920 0x7f726806ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f727c001eb0 tx=0x7f727c001da0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 msgr2=0x7f728c1f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728be00640 1 --2- 172.21.15.32:0/528907774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f728c1f4c40 0x7f728c1f50f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7284005040 tx=0x7f72840014a0 comp rx=0 tx=0).stop 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.970+0000 7f728b400640 1 -- 172.21.15.32:0/528907774 reap_dead start 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.971+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 shutdown_connections 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.971+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 >> 172.21.15.32:0/528907774 conn(0x7f728c0d13a0 msgr2=0x7f728c06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.971+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 shutdown_connections 2024-04-12T19:52:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:13.971+0000 7f728be00640 1 -- 172.21.15.32:0/528907774 wait complete. 2024-04-12T19:52:13.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-12T19:52:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.215+0000 7f5ec7e00640 1 Processor -- start 2024-04-12T19:52:14.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.215+0000 7f5ec7e00640 1 -- start start 2024-04-12T19:52:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815b410 0x7f5ec815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5ec815cf20 0x7f5ec815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec815cae0 con 0x7f5ec80cf530 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec7e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec80cd360 con 0x7f5ec815b410 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec80cd640 con 0x7f5ec815cf20 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815b410 0x7f5ec815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815b410 0x7f5ec815c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48642/0 (socket says 172.21.15.32:48642) 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.217+0000 7f5ec6a00640 1 -- 172.21.15.32:0/2385029443 learned_addr learned my addr 172.21.15.32:0/2385029443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.218+0000 7f5ec7400640 1 --2- 172.21.15.32:0/2385029443 >> v2:172.21.15.169:3300/0 conn(0x7f5ec815cf20 0x7f5ec815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.218+0000 7f5ec6a00640 1 -- 172.21.15.32:0/2385029443 >> v2:172.21.15.169:3300/0 conn(0x7f5ec815cf20 msgr2=0x7f5ec815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.218+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/2385029443 >> v2:172.21.15.169:3300/0 conn(0x7f5ec815cf20 0x7f5ec815d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.218+0000 7f5ec6a00640 1 -- 172.21.15.32:0/2385029443 >> v1:172.21.15.169:6789/0 conn(0x7f5ec80cf530 legacy=0x7f5ec80cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.218+0000 7f5ec6a00640 1 -- 172.21.15.32:0/2385029443 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec80cd920 con 0x7f5ec815b410 2024-04-12T19:52:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.219+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/2385029443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec815c5a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80ce210 tx=0x7f5eb802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98966dd515b226a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.219+0000 7f5ec5600640 1 -- 172.21.15.32:0/2385029443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb802e070 con 0x7f5ec815b410 2024-04-12T19:52:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec5600640 1 -- 172.21.15.32:0/2385029443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8037070 con 0x7f5ec815b410 2024-04-12T19:52:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec5600640 1 -- 172.21.15.32:0/2385029443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8031a00 con 0x7f5ec815b410 2024-04-12T19:52:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 -- 172.21.15.32:0/2385029443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 msgr2=0x7f5ec815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/2385029443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec815c5a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80ce210 tx=0x7f5eb802b880 comp rx=0 tx=0).stop 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 -- 172.21.15.32:0/2385029443 shutdown_connections 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/2385029443 >> v2:172.21.15.169:3300/0 conn(0x7f5ec815cf20 0x7f5ec815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/2385029443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec815c5a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.220+0000 7f5ec7e00640 1 -- 172.21.15.32:0/2385029443 >> 172.21.15.32:0/2385029443 conn(0x7f5ec80b23a0 msgr2=0x7f5ec80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.221+0000 7f5ec7e00640 1 -- 172.21.15.32:0/2385029443 shutdown_connections 2024-04-12T19:52:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.221+0000 7f5ec7e00640 1 -- 172.21.15.32:0/2385029443 wait complete. 2024-04-12T19:52:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.222+0000 7f5ec7e00640 1 Processor -- start 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.223+0000 7f5ec7e00640 1 -- start start 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.223+0000 7f5ec7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 0x7f5ec80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.223+0000 7f5ec7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec815cf20 0x7f5ec8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80cd030 con 0x7f5ec815b410 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80cdee0 con 0x7f5ec815cf20 2024-04-12T19:52:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec80b03a0 con 0x7f5ec80cf530 2024-04-12T19:52:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48654/0 (socket says 172.21.15.32:48654) 2024-04-12T19:52:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 -- 172.21.15.32:0/882609534 learned_addr learned my addr 172.21.15.32:0/882609534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 0x7f5ec80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec7400640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec815cf20 0x7f5ec8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 -- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 msgr2=0x7f5ec80b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 0x7f5ec80b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 -- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec815cf20 msgr2=0x7f5ec8150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec815cf20 0x7f5ec8150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.224+0000 7f5ec6000640 1 -- 172.21.15.32:0/882609534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec80b5f80 con 0x7f5ec815b410 2024-04-12T19:52:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 0x7f5ec80b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ec6000640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c430 tx=0x7f5eb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011790 con 0x7f5ec815b410 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4017070 con 0x7f5ec815b410 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec8150340 con 0x7f5ec815b410 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011bb0 con 0x7f5ec815b410 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.225+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec80b6260 con 0x7f5ec815b410 2024-04-12T19:52:14.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.226+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec80755c0 con 0x7f5ec815b410 2024-04-12T19:52:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.227+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5eb401c460 con 0x7f5ec815b410 2024-04-12T19:52:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.228+0000 7f5ebea00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.229+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f5eb408b6d0 con 0x7f5ec815b410 2024-04-12T19:52:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.229+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.229+0000 7f5ec6a00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 0x7f5ea806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8036000 tx=0x7f5eb803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.233+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5eb4059e00 con 0x7f5ec815b410 2024-04-12T19:52:14.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.436+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f5ec80cf9a0 con 0x7f5ea806c920 2024-04-12T19:52:14.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.436+0000 7f5ebea00640 1 -- 172.21.15.32:0/882609534 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5ec80cf9a0 con 0x7f5ea806c920 2024-04-12T19:52:14.437 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2024-04-12T19:52:14.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 msgr2=0x7f5ea806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 0x7f5ea806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8036000 tx=0x7f5eb803d040 comp rx=0 tx=0).stop 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 msgr2=0x7f5ec814efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c430 tx=0x7f5eb400c8f0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 shutdown_connections 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f5ea806c920 0x7f5ea806ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec815cf20 0x7f5ec8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815b410 0x7f5ec814efa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 --2- 172.21.15.32:0/882609534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ec80cf530 0x7f5ec80b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 >> 172.21.15.32:0/882609534 conn(0x7f5ec80b23a0 msgr2=0x7f5ec806f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:14.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 shutdown_connections 2024-04-12T19:52:14.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.439+0000 7f5ec7e00640 1 -- 172.21.15.32:0/882609534 wait complete. 2024-04-12T19:52:14.457 INFO:tasks.ceph:Scrubbing osd.4 2024-04-12T19:52:14.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:14.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.687+0000 7f71de000640 1 Processor -- start 2024-04-12T19:52:14.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.687+0000 7f71de000640 1 -- start start 2024-04-12T19:52:14.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.688+0000 7f71de000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80cf510 0x7f71d80b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.688+0000 7f71de000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71d80d2b90 0x7f71d80cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.688+0000 7f71de000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d80cfe30 con 0x7f71d806cd90 2024-04-12T19:52:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.688+0000 7f71de000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f71d80d3f60 con 0x7f71d80d2b90 2024-04-12T19:52:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.688+0000 7f71de000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80d3350 con 0x7f71d80cf510 2024-04-12T19:52:14.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71dd600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71d806cd90 0x7f71d806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50392/0 (socket says 172.21.15.32:50392) 2024-04-12T19:52:14.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71dd600640 1 -- 172.21.15.32:0/200404423 learned_addr learned my addr 172.21.15.32:0/200404423 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:14.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71d7e00640 1 --2- 172.21.15.32:0/200404423 >> v2:172.21.15.32:3300/0 conn(0x7f71d80d2b90 0x7f71d80cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71dcc00640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80cf510 0x7f71d80b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71d7400640 1 -- 172.21.15.32:0/200404423 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279173054 0 0) 0x7f71d80cfe30 con 0x7f71d806cd90 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.689+0000 7f71d7400640 1 -- 172.21.15.32:0/200404423 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71bc003520 con 0x7f71d806cd90 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.690+0000 7f71d7e00640 1 -- 172.21.15.32:0/200404423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80cf510 msgr2=0x7f71d80b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.690+0000 7f71d7e00640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80cf510 0x7f71d80b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.690+0000 7f71d7e00640 1 -- 172.21.15.32:0/200404423 >> v1:172.21.15.32:6789/0 conn(0x7f71d806cd90 legacy=0x7f71d806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.690+0000 7f71d7e00640 1 -- 172.21.15.32:0/200404423 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d80d3630 con 0x7f71d80d2b90 2024-04-12T19:52:14.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.691+0000 7f71d7e00640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d80d2b90 0x7f71d80cf8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f71c40097a0 tx=0x7f71c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5526533f39aeeccc server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.691+0000 7f71d7400640 1 -- 172.21.15.32:0/200404423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4035020 con 0x7f71d80d2b90 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71d7400640 1 -- 172.21.15.32:0/200404423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c4031810 con 0x7f71d80d2b90 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71d7400640 1 -- 172.21.15.32:0/200404423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4031bc0 con 0x7f71d80d2b90 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 -- 172.21.15.32:0/200404423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d80d2b90 msgr2=0x7f71d80cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d80d2b90 0x7f71d80cf8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f71c40097a0 tx=0x7f71c402b8b0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 -- 172.21.15.32:0/200404423 shutdown_connections 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d80d2b90 0x7f71d80cf8f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 --2- 172.21.15.32:0/200404423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80cf510 0x7f71d80b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 -- 172.21.15.32:0/200404423 >> 172.21.15.32:0/200404423 conn(0x7f71d80d13a0 msgr2=0x7f71d80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 -- 172.21.15.32:0/200404423 shutdown_connections 2024-04-12T19:52:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.692+0000 7f71de000640 1 -- 172.21.15.32:0/200404423 wait complete. 2024-04-12T19:52:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.694+0000 7f71de000640 1 Processor -- start 2024-04-12T19:52:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.694+0000 7f71de000640 1 -- start start 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 0x7f71d8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71d80cf510 0x7f71d8160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80d2b90 0x7f71d80cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80b88f0 con 0x7f71d806cd90 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80d3bf0 con 0x7f71d80cf510 2024-04-12T19:52:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71de000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80d2fb0 con 0x7f71d80d2b90 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 0x7f71d8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.695+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 0x7f71d8159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48666/0 (socket says 172.21.15.32:48666) 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 -- 172.21.15.32:0/1690833155 learned_addr learned my addr 172.21.15.32:0/1690833155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71d7e00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71d80cf510 0x7f71d8160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 -- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80d2b90 msgr2=0x7f71d80cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71d80d2b90 0x7f71d80cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 -- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71d80cf510 msgr2=0x7f71d8160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71d80cf510 0x7f71d8160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d80b73d0 con 0x7f71d806cd90 2024-04-12T19:52:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.696+0000 7f71dcc00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 0x7f71d8159ca0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e870 tx=0x7f71c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.697+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8017070 con 0x7f71d806cd90 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.697+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c800c940 con 0x7f71d806cd90 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.697+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800cd30 con 0x7f71d806cd90 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.697+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d80b6b30 con 0x7f71d806cd90 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.697+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d80b70f0 con 0x7f71d806cd90 2024-04-12T19:52:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.698+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71d8068b40 con 0x7f71d806cd90 2024-04-12T19:52:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.699+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c801a070 con 0x7f71d806cd90 2024-04-12T19:52:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d4c00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f71b806c920 0x7f71b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f71c808b730 con 0x7f71d806cd90 2024-04-12T19:52:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d7e00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f71b806c920 0x7f71b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d4c00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f71b80727e0 0x7f71b8074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71c8015e50 con 0x7f71b80727e0 2024-04-12T19:52:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.700+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f71c8092050 con 0x7f71d806cd90 2024-04-12T19:52:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.701+0000 7f71dd600640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f71b80727e0 0x7f71b8074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.701+0000 7f71d7e00640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f71b806c920 0x7f71b806ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71c4036000 tx=0x7f71c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.702+0000 7f71dd600640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f71b80727e0 0x7f71b8074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.705+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== osd.4 v2:172.21.15.169:6800/3921333402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71c8015e50 con 0x7f71b80727e0 2024-04-12T19:52:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.719+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 --> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71d815ef90 con 0x7f71b80727e0 2024-04-12T19:52:14.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71d4c00640 1 -- 172.21.15.32:0/1690833155 <== osd.4 v2:172.21.15.169:6800/3921333402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71d815ef90 con 0x7f71b80727e0 2024-04-12T19:52:14.721 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:14.721 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:14.721 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:14.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f71b80727e0 msgr2=0x7f71b8074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.169:6800/3921333402,v1:172.21.15.169:6801/3921333402] conn(0x7f71b80727e0 0x7f71b8074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f71b806c920 msgr2=0x7f71b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f71b806c920 0x7f71b806ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71c4036000 tx=0x7f71c4008030 comp rx=0 tx=0).stop 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 msgr2=0x7f71d8159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 --2- 172.21.15.32:0/1690833155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d806cd90 0x7f71d8159ca0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e870 tx=0x7f71c800ed30 comp rx=0 tx=0).stop 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71dd600640 1 -- 172.21.15.32:0/1690833155 reap_dead start 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 shutdown_connections 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.721+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 >> 172.21.15.32:0/1690833155 conn(0x7f71d80d13a0 msgr2=0x7f71d80c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.722+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 shutdown_connections 2024-04-12T19:52:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.722+0000 7f71de000640 1 -- 172.21.15.32:0/1690833155 wait complete. 2024-04-12T19:52:14.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-12T19:52:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.962+0000 7f33d9200640 1 Processor -- start 2024-04-12T19:52:14.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.963+0000 7f33d9200640 1 -- start start 2024-04-12T19:52:14.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.964+0000 7f33d9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33d40cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.964+0000 7f33d9200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d40cd940 con 0x7f33d40b59b0 2024-04-12T19:52:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.964+0000 7f33d9200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d40cdc20 con 0x7f33d40b0cb0 2024-04-12T19:52:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.964+0000 7f33d9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33d40ce550 con 0x7f33d415d770 2024-04-12T19:52:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.965+0000 7f33d3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33d40b59b0 0x7f33d415dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50402/0 (socket says 172.21.15.32:50402) 2024-04-12T19:52:14.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.965+0000 7f33d3e00640 1 -- 172.21.15.32:0/576958836 learned_addr learned my addr 172.21.15.32:0/576958836 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.965+0000 7f33d2a00640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33d40cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.965+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088217344 0 0) 0x7f33d40cd940 con 0x7f33d40b59b0 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.965+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33b8003520 con 0x7f33d40b59b0 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523321267 0 0) 0x7f33b8003520 con 0x7f33d40b59b0 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 msgr2=0x7f33d40cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33d40cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 >> v1:172.21.15.169:6789/0 conn(0x7f33d40b0cb0 legacy=0x7f33d40ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d40cdf00 con 0x7f33d40b59b0 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f33cc0019f0 con 0x7f33d40b59b0 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 >> v1:172.21.15.32:6789/0 conn(0x7f33d40b59b0 legacy=0x7f33d415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b80039e0 0x7f33b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b8004800 0x7f33b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33d40ce550 con 0x7f33b80039e0 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33b800baa0 con 0x7f33b8004800 2024-04-12T19:52:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33b800bca0 con 0x7f33d415d770 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d3400640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b80039e0 0x7f33b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.966+0000 7f33d2a00640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b8004800 0x7f33b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3e00640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3400640 1 -- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 msgr2=0x7f33b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3400640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33d415d770 0x7f33b80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3400640 1 -- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b8004800 msgr2=0x7f33b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3400640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b8004800 0x7f33b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.967+0000 7f33d3400640 1 -- 172.21.15.32:0/576958836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8003520 con 0x7f33b80039e0 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.968+0000 7f33d3400640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b80039e0 0x7f33b8003e00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33c4001720 tx=0x7f33b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84e1f48a099de49 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.968+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4001370 con 0x7f33b80039e0 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.968+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c40052d0 con 0x7f33b80039e0 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.968+0000 7f33d2000640 1 -- 172.21.15.32:0/576958836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4003830 con 0x7f33b80039e0 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 -- 172.21.15.32:0/576958836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b80039e0 msgr2=0x7f33b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 --2- 172.21.15.32:0/576958836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b80039e0 0x7f33b8003e00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f33c4001720 tx=0x7f33b8005830 comp rx=0 tx=0).stop 2024-04-12T19:52:14.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d3e00640 1 -- 172.21.15.32:0/576958836 reap_dead start 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 -- 172.21.15.32:0/576958836 shutdown_connections 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 -- 172.21.15.32:0/576958836 >> 172.21.15.32:0/576958836 conn(0x7f33d40b23a0 msgr2=0x7f33d40b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 -- 172.21.15.32:0/576958836 shutdown_connections 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.969+0000 7f33d9200640 1 -- 172.21.15.32:0/576958836 wait complete. 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.971+0000 7f33d9200640 1 Processor -- start 2024-04-12T19:52:14.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.971+0000 7f33d9200640 1 -- start start 2024-04-12T19:52:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b80039e0 0x7f33d4145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33d415b490 0x7f33d415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33d415e380 con 0x7f33d4151d50 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33d40b1090 con 0x7f33d415b490 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33d415e500 con 0x7f33b80039e0 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48686/0 (socket says 172.21.15.32:48686) 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 -- 172.21.15.32:0/1412702141 learned_addr learned my addr 172.21.15.32:0/1412702141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:14.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 -- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b80039e0 msgr2=0x7f33d4145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b80039e0 0x7f33d4145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 -- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33d415b490 msgr2=0x7f33d415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33d415b490 0x7f33d415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.972+0000 7f33d2a00640 1 -- 172.21.15.32:0/1412702141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d4146470 con 0x7f33d4151d50 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33d2a00640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33c0011630 tx=0x7f33c0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000f920 con 0x7f33d4151d50 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c000fac0 con 0x7f33d4151d50 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000fe70 con 0x7f33d4151d50 2024-04-12T19:52:14.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d41f7ba0 con 0x7f33d4151d50 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.973+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d41f8160 con 0x7f33d4151d50 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.974+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d40b3040 con 0x7f33d4151d50 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.975+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33c0015460 con 0x7f33d4151d50 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.976+0000 7f33cb400640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 0x7f33b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.976+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f33c008c150 con 0x7f33d4151d50 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.977+0000 7f33d3400640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 0x7f33b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:14.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.977+0000 7f33d3400640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 0x7f33b406ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33c4033000 tx=0x7f33c402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:14.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:14.981+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33d4146470 con 0x7f33d4151d50 2024-04-12T19:52:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.191+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f33d415aa70 con 0x7f33b406c920 2024-04-12T19:52:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.192+0000 7f33cb400640 1 -- 172.21.15.32:0/1412702141 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f33d415aa70 con 0x7f33b406c920 2024-04-12T19:52:15.193 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2024-04-12T19:52:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 msgr2=0x7f33b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 0x7f33b406ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33c4033000 tx=0x7f33c402f000 comp rx=0 tx=0).stop 2024-04-12T19:52:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 msgr2=0x7f33d41521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33c0011630 tx=0x7f33c0011af0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 shutdown_connections 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f33b406c920 0x7f33b406ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33d415b490 0x7f33d415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33d4151d50 0x7f33d41521d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 --2- 172.21.15.32:0/1412702141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b80039e0 0x7f33d4145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 >> 172.21.15.32:0/1412702141 conn(0x7f33d40b23a0 msgr2=0x7f33d406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 shutdown_connections 2024-04-12T19:52:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.195+0000 7f33d9200640 1 -- 172.21.15.32:0/1412702141 wait complete. 2024-04-12T19:52:15.215 INFO:tasks.ceph:Scrubbing osd.5 2024-04-12T19:52:15.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:15.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.442+0000 7fd352400640 1 Processor -- start 2024-04-12T19:52:15.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.442+0000 7fd352400640 1 -- start start 2024-04-12T19:52:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd352400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd34c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd352400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0d26e0 con 0x7fd34c147ca0 2024-04-12T19:52:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd352400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0d29c0 con 0x7fd34c0b1ac0 2024-04-12T19:52:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd352400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2ca0 con 0x7fd34c0b4eb0 2024-04-12T19:52:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd34be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd34c147ca0 0x7fd34c06cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50416/0 (socket says 172.21.15.32:50416) 2024-04-12T19:52:15.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.444+0000 7fd34be00640 1 -- 172.21.15.32:0/1109939763 learned_addr learned my addr 172.21.15.32:0/1109939763 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd351a00640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd34c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542483771 0 0) 0x7fd34c0d26e0 con 0x7fd34c147ca0 2024-04-12T19:52:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd334003520 con 0x7fd34c147ca0 2024-04-12T19:52:15.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2224869463 0 0) 0x7fd334003520 con 0x7fd34c147ca0 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 msgr2=0x7fd34c0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd34c0b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 >> v1:172.21.15.169:6789/0 conn(0x7fd34c0b1ac0 legacy=0x7fd34c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0d2f80 con 0x7fd34c147ca0 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7fd338001690 con 0x7fd34c147ca0 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 >> v1:172.21.15.32:6789/0 conn(0x7fd34c147ca0 legacy=0x7fd34c06cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.445+0000 7fd34b400640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3340039e0 0x7fd334003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34b400640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd334004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34b400640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd334004d40 0x7fd3340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2ca0 con 0x7fd3340039e0 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400bbb0 con 0x7fd334004d40 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd33400bdb0 con 0x7fd34c0b4eb0 2024-04-12T19:52:15.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd351000640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3340039e0 0x7fd334003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd351a00640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd334004d40 0x7fd3340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.446+0000 7fd34be00640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd334004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.447+0000 7fd351000640 1 -- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 msgr2=0x7fd334004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.447+0000 7fd351000640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c0b4eb0 0x7fd334004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.447+0000 7fd351000640 1 -- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd334004d40 msgr2=0x7fd3340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.447+0000 7fd351000640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd334004d40 0x7fd3340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.447+0000 7fd351000640 1 -- 172.21.15.32:0/1109939763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd334003520 con 0x7fd3340039e0 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.448+0000 7fd351000640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3340039e0 0x7fd334003e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd340001750 tx=0x7fd334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b6400a2774a002 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340001370 con 0x7fd3340039e0 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3400052d0 con 0x7fd3340039e0 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd34b400640 1 -- 172.21.15.32:0/1109939763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340003830 con 0x7fd3340039e0 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd352400640 1 -- 172.21.15.32:0/1109939763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3340039e0 msgr2=0x7fd334003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd352400640 1 --2- 172.21.15.32:0/1109939763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3340039e0 0x7fd334003e00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd340001750 tx=0x7fd334005830 comp rx=0 tx=0).stop 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd351a00640 1 -- 172.21.15.32:0/1109939763 reap_dead start 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd352400640 1 -- 172.21.15.32:0/1109939763 shutdown_connections 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.449+0000 7fd352400640 1 -- 172.21.15.32:0/1109939763 >> 172.21.15.32:0/1109939763 conn(0x7fd34c067b40 msgr2=0x7fd34c067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.450+0000 7fd352400640 1 -- 172.21.15.32:0/1109939763 shutdown_connections 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.450+0000 7fd352400640 1 -- 172.21.15.32:0/1109939763 wait complete. 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.451+0000 7fd352400640 1 Processor -- start 2024-04-12T19:52:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.452+0000 7fd352400640 1 -- start start 2024-04-12T19:52:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.452+0000 7fd352400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c13d1e0 0x7fd34c1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd352400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c13d5c0 0x7fd34c161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd352400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 0x7fd34c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd352400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0b2020 con 0x7fd34c0cd890 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd352400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0b1ea0 con 0x7fd34c13d1e0 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd352400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0d2270 con 0x7fd34c13d5c0 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 0x7fd34c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 0x7fd34c0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48704/0 (socket says 172.21.15.32:48704) 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 -- 172.21.15.32:0/2038942461 learned_addr learned my addr 172.21.15.32:0/2038942461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351000640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c13d1e0 0x7fd34c1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd34be00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c13d5c0 0x7fd34c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 -- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c13d5c0 msgr2=0x7fd34c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c13d5c0 0x7fd34c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 -- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c13d1e0 msgr2=0x7fd34c1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.453+0000 7fd351a00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c13d1e0 0x7fd34c1611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd351a00640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c13ebf0 con 0x7fd34c0cd890 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd351000640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd34c13d1e0 0x7fd34c1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd34be00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd34c13d5c0 0x7fd34c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd351a00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 0x7fd34c0cdd20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1486d0 tx=0x7fd34400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34400fe10 con 0x7fd34c0cd890 2024-04-12T19:52:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34401d070 con 0x7fd34c0cd890 2024-04-12T19:52:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344015670 con 0x7fd34c0cd890 2024-04-12T19:52:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c13eed0 con 0x7fd34c0cd890 2024-04-12T19:52:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.454+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c13f490 con 0x7fd34c0cd890 2024-04-12T19:52:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.455+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd34c161c60 con 0x7fd34c0cd890 2024-04-12T19:52:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.456+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd344015810 con 0x7fd34c0cd890 2024-04-12T19:52:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.457+0000 7fd348c00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd32c06c920 0x7fd32c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.458+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fd34408c130 con 0x7fd34c0cd890 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.458+0000 7fd351000640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd32c06c920 0x7fd32c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.458+0000 7fd348c00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fd32c0727e0 0x7fd32c074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.458+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd34400ce90 con 0x7fd32c0727e0 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.458+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd344093050 con 0x7fd34c0cd890 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.459+0000 7fd34be00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fd32c0727e0 0x7fd32c074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.459+0000 7fd351000640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd32c06c920 0x7fd32c06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd340033000 tx=0x7fd340009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.460+0000 7fd34be00640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fd32c0727e0 0x7fd32c074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.462+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== osd.5 v2:172.21.15.169:6816/2535062120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd34400ce90 con 0x7fd32c0727e0 2024-04-12T19:52:15.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.477+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 --> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd34c068b40 con 0x7fd32c0727e0 2024-04-12T19:52:15.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.478+0000 7fd348c00640 1 -- 172.21.15.32:0/2038942461 <== osd.5 v2:172.21.15.169:6816/2535062120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd34c068b40 con 0x7fd32c0727e0 2024-04-12T19:52:15.478 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:15.479 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:15.479 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fd32c0727e0 msgr2=0x7fd32c074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.169:6816/2535062120,v1:172.21.15.169:6817/2535062120] conn(0x7fd32c0727e0 0x7fd32c074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd32c06c920 msgr2=0x7fd32c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fd32c06c920 0x7fd32c06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd340033000 tx=0x7fd340009030 comp rx=0 tx=0).stop 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 msgr2=0x7fd34c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 --2- 172.21.15.32:0/2038942461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd34c0cd890 0x7fd34c0cdd20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd34c1486d0 tx=0x7fd34400f580 comp rx=0 tx=0).stop 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd351a00640 1 -- 172.21.15.32:0/2038942461 reap_dead start 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 shutdown_connections 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 >> 172.21.15.32:0/2038942461 conn(0x7fd34c067b40 msgr2=0x7fd34c0d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 shutdown_connections 2024-04-12T19:52:15.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.479+0000 7fd352400640 1 -- 172.21.15.32:0/2038942461 wait complete. 2024-04-12T19:52:15.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-12T19:52:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.706+0000 7faa1da00640 1 Processor -- start 2024-04-12T19:52:15.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.706+0000 7faa1da00640 1 -- start start 2024-04-12T19:52:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa1da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa180b0cb0 0x7faa180ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa1da00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa180cd940 con 0x7faa1815d770 2024-04-12T19:52:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa1da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa180cdc20 con 0x7faa180b59b0 2024-04-12T19:52:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa1da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faa180ce550 con 0x7faa180b0cb0 2024-04-12T19:52:15.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa17400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa1815d770 0x7faa180cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50432/0 (socket says 172.21.15.32:50432) 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa17400640 1 -- 172.21.15.32:0/3909328959 learned_addr learned my addr 172.21.15.32:0/3909328959 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.708+0000 7faa17e00640 1 --2- 172.21.15.32:0/3909328959 >> v2:172.21.15.169:3300/0 conn(0x7faa180b0cb0 0x7faa180ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622749194 0 0) 0x7faa180cd940 con 0x7faa1815d770 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9fc003520 con 0x7faa1815d770 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa17e00640 1 -- 172.21.15.32:0/3909328959 >> v1:172.21.15.169:6789/0 conn(0x7faa180b59b0 legacy=0x7faa1815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa17e00640 1 -- 172.21.15.32:0/3909328959 >> v1:172.21.15.32:6789/0 conn(0x7faa1815d770 legacy=0x7faa180cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2255459662 0 0) 0x7fa9fc003520 con 0x7faa1815d770 2024-04-12T19:52:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.709+0000 7faa17e00640 1 -- 172.21.15.32:0/3909328959 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa180cdf00 con 0x7faa180b0cb0 2024-04-12T19:52:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.711+0000 7faa17e00640 1 --2- 172.21.15.32:0/3909328959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa180b0cb0 0x7faa180ce9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa080097a0 tx=0x7faa0802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215533173bc62f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.711+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08038070 con 0x7faa180b0cb0 2024-04-12T19:52:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.711+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08031500 con 0x7faa180b0cb0 2024-04-12T19:52:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa16a00640 1 -- 172.21.15.32:0/3909328959 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04001670 con 0x7faa180b0cb0 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 -- 172.21.15.32:0/3909328959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa180b0cb0 msgr2=0x7faa180ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 --2- 172.21.15.32:0/3909328959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa180b0cb0 0x7faa180ce9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa080097a0 tx=0x7faa0802b820 comp rx=0 tx=0).stop 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 -- 172.21.15.32:0/3909328959 shutdown_connections 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 --2- 172.21.15.32:0/3909328959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa180b0cb0 0x7faa180ce9e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 -- 172.21.15.32:0/3909328959 >> 172.21.15.32:0/3909328959 conn(0x7faa180b23a0 msgr2=0x7faa180b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 -- 172.21.15.32:0/3909328959 shutdown_connections 2024-04-12T19:52:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.712+0000 7faa1da00640 1 -- 172.21.15.32:0/3909328959 wait complete. 2024-04-12T19:52:15.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.714+0000 7faa1da00640 1 Processor -- start 2024-04-12T19:52:15.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.714+0000 7faa1da00640 1 -- start start 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 0x7faa1815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 0x7faa1815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa180cef20 con 0x7faa180b0cb0 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa180b1090 con 0x7faa1815d770 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa1da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa1813dd70 con 0x7faa180b59b0 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.715+0000 7faa17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48706/0 (socket says 172.21.15.32:48706) 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 -- 172.21.15.32:0/1916523275 learned_addr learned my addr 172.21.15.32:0/1916523275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa1d000640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 0x7faa1815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17400640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 0x7faa1815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 -- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 msgr2=0x7faa1815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 0x7faa1815fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 -- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 msgr2=0x7faa1815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 0x7faa1815dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 -- 172.21.15.32:0/1916523275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1813e010 con 0x7faa180b0cb0 2024-04-12T19:52:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa1d000640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 0x7faa1815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17400640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 0x7faa1815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.716+0000 7faa17e00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa08036040 tx=0x7faa080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.717+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08043070 con 0x7faa180b0cb0 2024-04-12T19:52:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.717+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0803f870 con 0x7faa180b0cb0 2024-04-12T19:52:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.717+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa18160720 con 0x7faa180b0cb0 2024-04-12T19:52:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.717+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08038040 con 0x7faa180b0cb0 2024-04-12T19:52:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.717+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa18160ce0 con 0x7faa180b0cb0 2024-04-12T19:52:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.718+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa180755c0 con 0x7faa180b0cb0 2024-04-12T19:52:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.719+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa08046070 con 0x7faa180b0cb0 2024-04-12T19:52:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.720+0000 7faa0fe00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.721+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7faa080ad430 con 0x7faa180b0cb0 2024-04-12T19:52:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.721+0000 7faa17400640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.722+0000 7faa17400640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 0x7fa9f806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa18072de0 tx=0x7faa04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.725+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa1813e010 con 0x7faa180b0cb0 2024-04-12T19:52:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.932+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7faa18153940 con 0x7fa9f806c8f0 2024-04-12T19:52:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.933+0000 7faa0fe00640 1 -- 172.21.15.32:0/1916523275 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faa18153940 con 0x7fa9f806c8f0 2024-04-12T19:52:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2024-04-12T19:52:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 msgr2=0x7fa9f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 0x7fa9f806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa18072de0 tx=0x7faa04009040 comp rx=0 tx=0).stop 2024-04-12T19:52:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 msgr2=0x7faa1813d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faa08036040 tx=0x7faa080046c0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 shutdown_connections 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa9f806c8f0 0x7fa9f806edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa1815d770 0x7faa1815dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa180b59b0 0x7faa1815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 --2- 172.21.15.32:0/1916523275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa180b0cb0 0x7faa1813d830 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 >> 172.21.15.32:0/1916523275 conn(0x7faa180b23a0 msgr2=0x7faa180af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 shutdown_connections 2024-04-12T19:52:15.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:15.936+0000 7faa1da00640 1 -- 172.21.15.32:0/1916523275 wait complete. 2024-04-12T19:52:15.955 INFO:tasks.ceph:Scrubbing osd.6 2024-04-12T19:52:15.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.164+0000 7f1bb5c00640 1 Processor -- start 2024-04-12T19:52:16.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.164+0000 7f1bb5c00640 1 -- start start 2024-04-12T19:52:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.165+0000 7f1bb5c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1bb00cec60 0x7f1bb0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.166+0000 7f1bb5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 0x7f1bb006f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.166+0000 7f1bb5c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bb013fc70 con 0x7f1bb0072450 2024-04-12T19:52:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.166+0000 7f1bb5c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1bb00728c0 con 0x7f1bb00cec60 2024-04-12T19:52:16.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.166+0000 7f1bb5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb013f9e0 con 0x7f1bb015d420 2024-04-12T19:52:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.166+0000 7f1bafe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1bb00cec60 0x7f1bb0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.167+0000 7f1bb5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 0x7f1bb006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.167+0000 7f1bafe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1bb00cec60 0x7f1bb0071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.32:40270/0 (socket says 172.21.15.32:40270) 2024-04-12T19:52:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.167+0000 7f1bafe00640 1 -- 172.21.15.32:0/4109734589 learned_addr learned my addr 172.21.15.32:0/4109734589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:16.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.167+0000 7f1baf400640 1 --1- 172.21.15.32:0/4109734589 >> v1:172.21.15.169:6789/0 conn(0x7f1bb0072450 0x7f1bb015cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.32:59344/0 (socket says 172.21.15.32:59344) 2024-04-12T19:52:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1bafe00640 1 -- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 msgr2=0x7f1bb006f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1bafe00640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 0x7f1bb006f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1bafe00640 1 -- 172.21.15.32:0/4109734589 >> v1:172.21.15.169:6789/0 conn(0x7f1bb0072450 legacy=0x7f1bb015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1bafe00640 1 -- 172.21.15.32:0/4109734589 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb0141a00 con 0x7f1bb00cec60 2024-04-12T19:52:16.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1baea00640 1 -- 172.21.15.32:0/4109734589 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393945105 0 0) 0x7f1bb013fc70 con 0x7f1bb0072450 2024-04-12T19:52:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.168+0000 7f1bb5200640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 0x7f1bb006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T19:52:16.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.169+0000 7f1bafe00640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00cec60 0x7f1bb0071f10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00098a0 tx=0x7f1ba002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532b406fac7b0aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.170+0000 7f1baea00640 1 -- 172.21.15.32:0/4109734589 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba002f830 con 0x7f1bb00cec60 2024-04-12T19:52:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.170+0000 7f1baea00640 1 -- 172.21.15.32:0/4109734589 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba002f9d0 con 0x7f1bb00cec60 2024-04-12T19:52:16.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.170+0000 7f1baea00640 1 -- 172.21.15.32:0/4109734589 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba002fda0 con 0x7f1bb00cec60 2024-04-12T19:52:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 -- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00cec60 msgr2=0x7f1bb0071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00cec60 0x7f1bb0071f10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00098a0 tx=0x7f1ba002ba10 comp rx=0 tx=0).stop 2024-04-12T19:52:16.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 -- 172.21.15.32:0/4109734589 shutdown_connections 2024-04-12T19:52:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb015d420 0x7f1bb006f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/4109734589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00cec60 0x7f1bb0071f10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 -- 172.21.15.32:0/4109734589 >> 172.21.15.32:0/4109734589 conn(0x7f1bb00b6170 msgr2=0x7f1bb006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 -- 172.21.15.32:0/4109734589 shutdown_connections 2024-04-12T19:52:16.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.171+0000 7f1bb5c00640 1 -- 172.21.15.32:0/4109734589 wait complete. 2024-04-12T19:52:16.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.173+0000 7f1bb5c00640 1 Processor -- start 2024-04-12T19:52:16.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.173+0000 7f1bb5c00640 1 -- start start 2024-04-12T19:52:16.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 0x7f1bb013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb00cec60 0x7f1bb0152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb015d420 0x7f1bb013a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01401f0 con 0x7f1bb0072450 2024-04-12T19:52:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb013ff10 con 0x7f1bb015d420 2024-04-12T19:52:16.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb01404d0 con 0x7f1bb00cec60 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 0x7f1bb013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 0x7f1bb013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48716/0 (socket says 172.21.15.32:48716) 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bafe00640 1 -- 172.21.15.32:0/504650439 learned_addr learned my addr 172.21.15.32:0/504650439 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1bb5200640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb015d420 0x7f1bb013a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.175+0000 7f1baf400640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb00cec60 0x7f1bb0152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 -- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb00cec60 msgr2=0x7f1bb0152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bb00cec60 0x7f1bb0152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 -- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb015d420 msgr2=0x7f1bb013a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb015d420 0x7f1bb013a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb01551a0 con 0x7f1bb0072450 2024-04-12T19:52:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bafe00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 0x7f1bb013e400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099c0 tx=0x7f1ba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0037c40 con 0x7f1bb0072450 2024-04-12T19:52:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba0046070 con 0x7f1bb0072450 2024-04-12T19:52:16.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0041400 con 0x7f1bb0072450 2024-04-12T19:52:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.176+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb0155480 con 0x7f1bb0072450 2024-04-12T19:52:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.177+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01530a0 con 0x7f1bb0072450 2024-04-12T19:52:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.178+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bb0068b40 con 0x7f1bb0072450 2024-04-12T19:52:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.179+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ba0040070 con 0x7f1bb0072450 2024-04-12T19:52:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1ba7e00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1b8c06c8f0 0x7f1b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f1ba00ae0c0 con 0x7f1bb0072450 2024-04-12T19:52:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1baf400640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1b8c06c8f0 0x7f1b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1ba7e00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1b8c0727b0 0x7f1b8c074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ba0008f00 con 0x7f1b8c0727b0 2024-04-12T19:52:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.180+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1ba00b5050 con 0x7f1bb0072450 2024-04-12T19:52:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.181+0000 7f1baf400640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1b8c06c8f0 0x7f1b8c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005fd0 tx=0x7f1b98007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.181+0000 7f1bb5200640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1b8c0727b0 0x7f1b8c074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.182+0000 7f1bb5200640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1b8c0727b0 0x7f1b8c074bd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.184+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== osd.6 v2:172.21.15.169:6808/2726062170 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ba0008f00 con 0x7f1b8c0727b0 2024-04-12T19:52:16.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.199+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 --> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1bb00730e0 con 0x7f1b8c0727b0 2024-04-12T19:52:16.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1ba7e00640 1 -- 172.21.15.32:0/504650439 <== osd.6 v2:172.21.15.169:6808/2726062170 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1bb00730e0 con 0x7f1b8c0727b0 2024-04-12T19:52:16.200 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:16.200 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:16.200 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:16.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1b8c0727b0 msgr2=0x7f1b8c074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.169:6808/2726062170,v1:172.21.15.169:6809/2726062170] conn(0x7f1b8c0727b0 0x7f1b8c074bd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1b8c06c8f0 msgr2=0x7f1b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f1b8c06c8f0 0x7f1b8c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005fd0 tx=0x7f1b98007f10 comp rx=0 tx=0).stop 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 msgr2=0x7f1bb013e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5c00640 1 --2- 172.21.15.32:0/504650439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1bb0072450 0x7f1bb013e400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099c0 tx=0x7f1ba00046c0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.200+0000 7f1bb5200640 1 -- 172.21.15.32:0/504650439 reap_dead start 2024-04-12T19:52:16.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.201+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 shutdown_connections 2024-04-12T19:52:16.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.201+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 >> 172.21.15.32:0/504650439 conn(0x7f1bb00b6170 msgr2=0x7f1bb0074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.201+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 shutdown_connections 2024-04-12T19:52:16.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.201+0000 7f1bb5c00640 1 -- 172.21.15.32:0/504650439 wait complete. 2024-04-12T19:52:16.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-12T19:52:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.443+0000 7efd36e00640 1 Processor -- start 2024-04-12T19:52:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.443+0000 7efd36e00640 1 -- start start 2024-04-12T19:52:16.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.445+0000 7efd36e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd300b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.445+0000 7efd36e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd300cd940 con 0x7efd3015d770 2024-04-12T19:52:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.445+0000 7efd36e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd300cdc20 con 0x7efd300b0cb0 2024-04-12T19:52:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.445+0000 7efd36e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd300ce550 con 0x7efd300b59b0 2024-04-12T19:52:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.446+0000 7efd35000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd3015d770 0x7efd300cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50436/0 (socket says 172.21.15.32:50436) 2024-04-12T19:52:16.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.446+0000 7efd35000640 1 -- 172.21.15.32:0/4289596514 learned_addr learned my addr 172.21.15.32:0/4289596514 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.446+0000 7efd36400640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd300b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.446+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732962222 0 0) 0x7efd300cd940 con 0x7efd3015d770 2024-04-12T19:52:16.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.446+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd18003520 con 0x7efd3015d770 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242476330 0 0) 0x7efd18003520 con 0x7efd3015d770 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 msgr2=0x7efd300b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd300b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 >> v1:172.21.15.169:6789/0 conn(0x7efd300b0cb0 legacy=0x7efd300ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd300cdf00 con 0x7efd3015d770 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7efd20001670 con 0x7efd3015d770 2024-04-12T19:52:16.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 >> v1:172.21.15.32:6789/0 conn(0x7efd3015d770 legacy=0x7efd300cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd18003f20 0x7efd18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd300ce550 con 0x7efd18003f20 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800ba20 con 0x7efd18004d40 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd1800bc20 con 0x7efd300b59b0 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd35000640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd18003f20 0x7efd18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd35a00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.447+0000 7efd36400640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.448+0000 7efd35000640 1 -- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 msgr2=0x7efd180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.448+0000 7efd35000640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd300b59b0 0x7efd180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.448+0000 7efd35000640 1 -- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18004d40 msgr2=0x7efd180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.448+0000 7efd35000640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd18004d40 0x7efd180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.448+0000 7efd35000640 1 -- 172.21.15.32:0/4289596514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd300cdf00 con 0x7efd18003f20 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.449+0000 7efd35000640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd18003f20 0x7efd18004800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c210 tx=0x7efd18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f3a57148185368 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.449+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20007d30 con 0x7efd18003f20 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.449+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd20030070 con 0x7efd18003f20 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.449+0000 7efd2fe00640 1 -- 172.21.15.32:0/4289596514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20005cf0 con 0x7efd18003f20 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 -- 172.21.15.32:0/4289596514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd18003f20 msgr2=0x7efd18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 --2- 172.21.15.32:0/4289596514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd18003f20 0x7efd18004800 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c210 tx=0x7efd18005830 comp rx=0 tx=0).stop 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36400640 1 -- 172.21.15.32:0/4289596514 reap_dead start 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 -- 172.21.15.32:0/4289596514 shutdown_connections 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 -- 172.21.15.32:0/4289596514 >> 172.21.15.32:0/4289596514 conn(0x7efd300b23a0 msgr2=0x7efd300b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 -- 172.21.15.32:0/4289596514 shutdown_connections 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.450+0000 7efd36e00640 1 -- 172.21.15.32:0/4289596514 wait complete. 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.452+0000 7efd36e00640 1 Processor -- start 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.452+0000 7efd36e00640 1 -- start start 2024-04-12T19:52:16.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd18003f20 0x7efd3013ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 0x7efd300b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd300b5e60 con 0x7efd300b3c70 2024-04-12T19:52:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd300b1090 con 0x7efd3014d320 2024-04-12T19:52:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd300b5fe0 con 0x7efd18003f20 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48734/0 (socket says 172.21.15.32:48734) 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd36400640 1 -- 172.21.15.32:0/2057238236 learned_addr learned my addr 172.21.15.32:0/2057238236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.453+0000 7efd35000640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 0x7efd300b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 -- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd18003f20 msgr2=0x7efd3013ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd18003f20 0x7efd3013ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 -- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 msgr2=0x7efd300b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 0x7efd300b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 -- 172.21.15.32:0/2057238236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3015dc10 con 0x7efd300b3c70 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd35000640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 0x7efd300b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36400640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efd3015db50 tx=0x7efd280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28011e10 con 0x7efd300b3c70 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd2801b070 con 0x7efd300b3c70 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28003990 con 0x7efd300b3c70 2024-04-12T19:52:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3015def0 con 0x7efd300b3c70 2024-04-12T19:52:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.454+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301e6420 con 0x7efd300b3c70 2024-04-12T19:52:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.456+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd30069280 con 0x7efd300b3c70 2024-04-12T19:52:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.457+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd28005d10 con 0x7efd300b3c70 2024-04-12T19:52:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.458+0000 7efd2d600640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 0x7efd1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.458+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7efd2808bee0 con 0x7efd300b3c70 2024-04-12T19:52:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.458+0000 7efd35a00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 0x7efd1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.459+0000 7efd35a00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 0x7efd1406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.463+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd30069280 con 0x7efd300b3c70 2024-04-12T19:52:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.665+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efd301e6210 con 0x7efd1406c8f0 2024-04-12T19:52:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.666+0000 7efd2d600640 1 -- 172.21.15.32:0/2057238236 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd301e6210 con 0x7efd1406c8f0 2024-04-12T19:52:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2024-04-12T19:52:16.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 msgr2=0x7efd1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 0x7efd1406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24009040 comp rx=0 tx=0).stop 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 msgr2=0x7efd300b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efd3015db50 tx=0x7efd280115a0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 shutdown_connections 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd1406c8f0 0x7efd1406edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd300b3c70 0x7efd300b4100 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3014d320 0x7efd300b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 --2- 172.21.15.32:0/2057238236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd18003f20 0x7efd3013ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 >> 172.21.15.32:0/2057238236 conn(0x7efd300b23a0 msgr2=0x7efd30151990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 shutdown_connections 2024-04-12T19:52:16.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.669+0000 7efd36e00640 1 -- 172.21.15.32:0/2057238236 wait complete. 2024-04-12T19:52:16.688 INFO:tasks.ceph:Scrubbing osd.7 2024-04-12T19:52:16.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-12T19:52:16.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.914+0000 7fc505a00640 1 Processor -- start 2024-04-12T19:52:16.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.914+0000 7fc505a00640 1 -- start start 2024-04-12T19:52:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.915+0000 7fc505a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000d25d0 0x7fc50015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.915+0000 7fc505a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 0x7fc5000b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.915+0000 7fc505a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc50006d410 con 0x7fc5000b6700 2024-04-12T19:52:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.915+0000 7fc505a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc50006e820 con 0x7fc5000d25d0 2024-04-12T19:52:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.915+0000 7fc505a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc50006d6f0 con 0x7fc5000d29b0 2024-04-12T19:52:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000d25d0 0x7fc50015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5000d25d0 0x7fc50015ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48744/0 (socket says 172.21.15.32:48744) 2024-04-12T19:52:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 -- 172.21.15.32:0/3692342739 learned_addr learned my addr 172.21.15.32:0/3692342739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 -- 172.21.15.32:0/3692342739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 msgr2=0x7fc5000b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 --2- 172.21.15.32:0/3692342739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 0x7fc5000b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 -- 172.21.15.32:0/3692342739 >> v1:172.21.15.32:6789/0 conn(0x7fc5000b6700 legacy=0x7fc50015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.916+0000 7fc4ffe00640 1 -- 172.21.15.32:0/3692342739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50006d9d0 con 0x7fc5000d25d0 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.917+0000 7fc4ffe00640 1 --2- 172.21.15.32:0/3692342739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000d25d0 0x7fc50015ffc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009870 tx=0x7fc4f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0cdb7b060b25cb server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.918+0000 7fc4fea00640 1 -- 172.21.15.32:0/3692342739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0035020 con 0x7fc5000d25d0 2024-04-12T19:52:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.918+0000 7fc4fea00640 1 -- 172.21.15.32:0/3692342739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0031640 con 0x7fc5000d25d0 2024-04-12T19:52:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.918+0000 7fc4fea00640 1 -- 172.21.15.32:0/3692342739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f002d040 con 0x7fc5000d25d0 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.918+0000 7fc505a00640 1 -- 172.21.15.32:0/3692342739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000d25d0 msgr2=0x7fc50015ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.918+0000 7fc505a00640 1 --2- 172.21.15.32:0/3692342739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000d25d0 0x7fc50015ffc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009870 tx=0x7fc4f002b8f0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 -- 172.21.15.32:0/3692342739 shutdown_connections 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 --2- 172.21.15.32:0/3692342739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 0x7fc5000b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 --2- 172.21.15.32:0/3692342739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000d25d0 0x7fc50015ffc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 -- 172.21.15.32:0/3692342739 >> 172.21.15.32:0/3692342739 conn(0x7fc500067de0 msgr2=0x7fc5000b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 -- 172.21.15.32:0/3692342739 shutdown_connections 2024-04-12T19:52:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.919+0000 7fc505a00640 1 -- 172.21.15.32:0/3692342739 wait complete. 2024-04-12T19:52:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.921+0000 7fc505a00640 1 Processor -- start 2024-04-12T19:52:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.921+0000 7fc505a00640 1 -- start start 2024-04-12T19:52:16.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 0x7fc50014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000d25d0 0x7fc5001576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 0x7fc5000d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc50015eb90 con 0x7fc5000b6700 2024-04-12T19:52:16.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc50006df90 con 0x7fc5000d25d0 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc505a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000d2e10 con 0x7fc5000d29b0 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 0x7fc50014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 0x7fc50014bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48752/0 (socket says 172.21.15.32:48752) 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.922+0000 7fc4ffe00640 1 -- 172.21.15.32:0/4009140074 learned_addr learned my addr 172.21.15.32:0/4009140074 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ff400640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000d25d0 0x7fc5001576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 -- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 msgr2=0x7fc5000d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000d29b0 0x7fc5000d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 -- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000d25d0 msgr2=0x7fc5001576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000d25d0 0x7fc5001576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000d0b60 con 0x7fc5000b6700 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ff400640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5000d25d0 0x7fc5001576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:16.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4ffe00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 0x7fc50014bfc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009870 tx=0x7fc4f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f003ac60 con 0x7fc5000b6700 2024-04-12T19:52:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f003ae20 con 0x7fc5000b6700 2024-04-12T19:52:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5000d0e40 con 0x7fc5000b6700 2024-04-12T19:52:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.923+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0044630 con 0x7fc5000b6700 2024-04-12T19:52:16.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.924+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5000d1400 con 0x7fc5000b6700 2024-04-12T19:52:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.925+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5000d17e0 con 0x7fc5000b6700 2024-04-12T19:52:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.926+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4f004b060 con 0x7fc5000b6700 2024-04-12T19:52:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4f7e00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4e006c940 0x7fc4e006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fc4f00ad500 con 0x7fc5000b6700 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4f7e00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7fc4e0072800 0x7fc4e0074c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4f0008f00 con 0x7fc4e0072800 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc4f00b4050 con 0x7fc5000b6700 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.927+0000 7fc4ff400640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4e006c940 0x7fc4e006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.928+0000 7fc4ff400640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4e006c940 0x7fc4e006ee00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5001f8050 tx=0x7fc4ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.928+0000 7fc505000640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7fc4e0072800 0x7fc4e0074c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.929+0000 7fc505000640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7fc4e0072800 0x7fc4e0074c20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:16.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.931+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== osd.7 v2:172.21.15.169:6822/883923800 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4f0008f00 con 0x7fc4e0072800 2024-04-12T19:52:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.946+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 --> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc5000755c0 con 0x7fc4e0072800 2024-04-12T19:52:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.947+0000 7fc4f7e00640 1 -- 172.21.15.32:0/4009140074 <== osd.7 v2:172.21.15.169:6822/883923800 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc5000755c0 con 0x7fc4e0072800 2024-04-12T19:52:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-12T19:52:16.947 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-12T19:52:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-12T19:52:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.947+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7fc4e0072800 msgr2=0x7fc4e0074c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.947+0000 7fc505a00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.169:6822/883923800,v1:172.21.15.169:6824/883923800] conn(0x7fc4e0072800 0x7fc4e0074c20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4e006c940 msgr2=0x7fc4e006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fc4e006c940 0x7fc4e006ee00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5001f8050 tx=0x7fc4ec008040 comp rx=0 tx=0).stop 2024-04-12T19:52:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 msgr2=0x7fc50014bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 --2- 172.21.15.32:0/4009140074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5000b6700 0x7fc50014bfc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009870 tx=0x7fc4f00046c0 comp rx=0 tx=0).stop 2024-04-12T19:52:16.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505000640 1 -- 172.21.15.32:0/4009140074 reap_dead start 2024-04-12T19:52:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 shutdown_connections 2024-04-12T19:52:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 >> 172.21.15.32:0/4009140074 conn(0x7fc500067de0 msgr2=0x7fc500161970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 shutdown_connections 2024-04-12T19:52:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:16.948+0000 7fc505a00640 1 -- 172.21.15.32:0/4009140074 wait complete. 2024-04-12T19:52:16.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-12T19:52:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.177+0000 7efd7e400640 1 Processor -- start 2024-04-12T19:52:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.177+0000 7efd7e400640 1 -- start start 2024-04-12T19:52:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.179+0000 7efd7e400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd780cf530 0x7efd780cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.179+0000 7efd7e400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780cd360 con 0x7efd7815cf20 2024-04-12T19:52:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.179+0000 7efd7e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780cd640 con 0x7efd7815b410 2024-04-12T19:52:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.179+0000 7efd7e400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efd780cd920 con 0x7efd780cf530 2024-04-12T19:52:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd77e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd780cf530 0x7efd780cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd7da00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd7815cf20 0x7efd7815d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50452/0 (socket says 172.21.15.32:50452) 2024-04-12T19:52:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd7da00640 1 -- 172.21.15.32:0/3183215157 learned_addr learned my addr 172.21.15.32:0/3183215157 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd77e00640 1 -- 172.21.15.32:0/3183215157 >> v1:172.21.15.169:6789/0 conn(0x7efd7815b410 legacy=0x7efd7815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd77e00640 1 -- 172.21.15.32:0/3183215157 >> v1:172.21.15.32:6789/0 conn(0x7efd7815cf20 legacy=0x7efd7815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd77400640 1 -- 172.21.15.32:0/3183215157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879236997 0 0) 0x7efd780cd360 con 0x7efd7815cf20 2024-04-12T19:52:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.180+0000 7efd77e00640 1 -- 172.21.15.32:0/3183215157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780cdc00 con 0x7efd780cf530 2024-04-12T19:52:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd77e00640 1 --2- 172.21.15.32:0/3183215157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd780cf530 0x7efd780cf9a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13092d4732d2c0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd77400640 1 -- 172.21.15.32:0/3183215157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64035020 con 0x7efd780cf530 2024-04-12T19:52:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd77400640 1 -- 172.21.15.32:0/3183215157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd64031830 con 0x7efd780cf530 2024-04-12T19:52:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd77400640 1 -- 172.21.15.32:0/3183215157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64031c00 con 0x7efd780cf530 2024-04-12T19:52:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd7e400640 1 -- 172.21.15.32:0/3183215157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd780cf530 msgr2=0x7efd780cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.182+0000 7efd7e400640 1 --2- 172.21.15.32:0/3183215157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd780cf530 0x7efd780cf9a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402b8d0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.183+0000 7efd7e400640 1 -- 172.21.15.32:0/3183215157 shutdown_connections 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.183+0000 7efd7e400640 1 --2- 172.21.15.32:0/3183215157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd780cf530 0x7efd780cf9a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.183+0000 7efd7e400640 1 -- 172.21.15.32:0/3183215157 >> 172.21.15.32:0/3183215157 conn(0x7efd780b23a0 msgr2=0x7efd780b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.183+0000 7efd7e400640 1 -- 172.21.15.32:0/3183215157 shutdown_connections 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.183+0000 7efd7e400640 1 -- 172.21.15.32:0/3183215157 wait complete. 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.184+0000 7efd7e400640 1 Processor -- start 2024-04-12T19:52:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.185+0000 7efd7e400640 1 -- start start 2024-04-12T19:52:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.185+0000 7efd7e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd780cf530 0x7efd780ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.185+0000 7efd7e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 0x7efd78152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd780cd030 con 0x7efd7815cf20 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd780cfee0 con 0x7efd780cf530 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd780b03a0 con 0x7efd7815b410 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48756/0 (socket says 172.21.15.32:48756) 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 -- 172.21.15.32:0/545105256 learned_addr learned my addr 172.21.15.32:0/545105256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7d000640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd780cf530 0x7efd780ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd77e00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 0x7efd78152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 -- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 msgr2=0x7efd78152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 0x7efd78152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 -- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd780cf530 msgr2=0x7efd780ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd780cf530 0x7efd780ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd7da00640 1 -- 172.21.15.32:0/545105256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd78153270 con 0x7efd7815cf20 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.186+0000 7efd77e00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 0x7efd78152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd7da00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c700 tx=0x7efd7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018070 con 0x7efd7815cf20 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7000ce10 con 0x7efd7815cf20 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7000a860 con 0x7efd7815cf20 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780adfe0 con 0x7efd7815cf20 2024-04-12T19:52:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.187+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd780ae5a0 con 0x7efd7815cf20 2024-04-12T19:52:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.188+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd780755c0 con 0x7efd7815cf20 2024-04-12T19:52:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.189+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd7001b070 con 0x7efd7815cf20 2024-04-12T19:52:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.191+0000 7efd74c00640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 0x7efd5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.191+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7efd7008ad30 con 0x7efd7815cf20 2024-04-12T19:52:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.191+0000 7efd7d000640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 0x7efd5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.192+0000 7efd7d000640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 0x7efd5806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000f60 tx=0x7efd6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.196+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd700594f0 con 0x7efd7815cf20 2024-04-12T19:52:17.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.404+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7efd7815aa20 con 0x7efd5806c8f0 2024-04-12T19:52:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.404+0000 7efd74c00640 1 -- 172.21.15.32:0/545105256 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd7815aa20 con 0x7efd5806c8f0 2024-04-12T19:52:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2024-04-12T19:52:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.407+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 msgr2=0x7efd5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.407+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 0x7efd5806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000f60 tx=0x7efd6c009040 comp rx=0 tx=0).stop 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 msgr2=0x7efd78150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c700 tx=0x7efd7000cbc0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 shutdown_connections 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efd5806c8f0 0x7efd5806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd7815cf20 0x7efd78150150 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd7815b410 0x7efd78152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 --2- 172.21.15.32:0/545105256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd780cf530 0x7efd780ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 >> 172.21.15.32:0/545105256 conn(0x7efd780b23a0 msgr2=0x7efd7815f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 shutdown_connections 2024-04-12T19:52:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.408+0000 7efd7e400640 1 -- 172.21.15.32:0/545105256 wait complete. 2024-04-12T19:52:17.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-12T19:52:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.655+0000 7fa835c00640 1 Processor -- start 2024-04-12T19:52:17.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.655+0000 7fa835c00640 1 -- start start 2024-04-12T19:52:17.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.656+0000 7fa835c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8300b6ea0 0x7fa8300b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa835c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa83006ca40 con 0x7fa83015e7e0 2024-04-12T19:52:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa835c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa83006cce0 con 0x7fa83009aa40 2024-04-12T19:52:17.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa835c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa8300b72f0 con 0x7fa8300b6ea0 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa82f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8300b6ea0 0x7fa8300b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa82fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa83015e7e0 0x7fa83006dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50456/0 (socket says 172.21.15.32:50456) 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.657+0000 7fa82fe00640 1 -- 172.21.15.32:0/3762662723 learned_addr learned my addr 172.21.15.32:0/3762662723 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.658+0000 7fa82f400640 1 -- 172.21.15.32:0/3762662723 >> v1:172.21.15.169:6789/0 conn(0x7fa83009aa40 legacy=0x7fa8300b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.658+0000 7fa82f400640 1 -- 172.21.15.32:0/3762662723 >> v1:172.21.15.32:6789/0 conn(0x7fa83015e7e0 legacy=0x7fa83006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.658+0000 7fa82f400640 1 -- 172.21.15.32:0/3762662723 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83006cfc0 con 0x7fa8300b6ea0 2024-04-12T19:52:17.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.659+0000 7fa82f400640 1 --2- 172.21.15.32:0/3762662723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300b6ea0 0x7fa8300b5eb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d90 tx=0x7fa81802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec00a9cf0f19c70d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.659+0000 7fa82ea00640 1 -- 172.21.15.32:0/3762662723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818039070 con 0x7fa8300b6ea0 2024-04-12T19:52:17.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.659+0000 7fa82ea00640 1 -- 172.21.15.32:0/3762662723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81802cc60 con 0x7fa8300b6ea0 2024-04-12T19:52:17.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.659+0000 7fa82ea00640 1 -- 172.21.15.32:0/3762662723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818034660 con 0x7fa8300b6ea0 2024-04-12T19:52:17.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 -- 172.21.15.32:0/3762662723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300b6ea0 msgr2=0x7fa8300b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 --2- 172.21.15.32:0/3762662723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300b6ea0 0x7fa8300b5eb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d90 tx=0x7fa81802beb0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 -- 172.21.15.32:0/3762662723 shutdown_connections 2024-04-12T19:52:17.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 --2- 172.21.15.32:0/3762662723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300b6ea0 0x7fa8300b5eb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 -- 172.21.15.32:0/3762662723 >> 172.21.15.32:0/3762662723 conn(0x7fa8300b40a0 msgr2=0x7fa8300b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 -- 172.21.15.32:0/3762662723 shutdown_connections 2024-04-12T19:52:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.660+0000 7fa835c00640 1 -- 172.21.15.32:0/3762662723 wait complete. 2024-04-12T19:52:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.662+0000 7fa835c00640 1 Processor -- start 2024-04-12T19:52:17.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.662+0000 7fa835c00640 1 -- start start 2024-04-12T19:52:17.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa83009aa40 0x7fa8301541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8300b6ea0 0x7fa8301f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa83006d580 con 0x7fa83015e7e0 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa83006d8f0 con 0x7fa83009aa40 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.663+0000 7fa835c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa83015ebc0 con 0x7fa8300b6ea0 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48776/0 (socket says 172.21.15.32:48776) 2024-04-12T19:52:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 -- 172.21.15.32:0/3539393227 learned_addr learned my addr 172.21.15.32:0/3539393227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa82fe00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa83009aa40 0x7fa8301541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa82f400640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8300b6ea0 0x7fa8301f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 -- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8300b6ea0 msgr2=0x7fa8301f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8300b6ea0 0x7fa8301f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 -- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa83009aa40 msgr2=0x7fa8301541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa83009aa40 0x7fa8301541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.664+0000 7fa835200640 1 -- 172.21.15.32:0/3539393227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8301f57d0 con 0x7fa83015e7e0 2024-04-12T19:52:17.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa835200640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b410 tx=0x7fa82800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828015030 con 0x7fa83015e7e0 2024-04-12T19:52:17.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa828004020 con 0x7fa83015e7e0 2024-04-12T19:52:17.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828004410 con 0x7fa83015e7e0 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8301f5ab0 con 0x7fa83015e7e0 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.665+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301e8750 con 0x7fa83015e7e0 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.667+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8301e8260 con 0x7fa83015e7e0 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.667+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa828018070 con 0x7fa83015e7e0 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.668+0000 7fa827e00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 0x7fa80c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:17.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.668+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa82808b2d0 con 0x7fa83015e7e0 2024-04-12T19:52:17.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.669+0000 7fa82fe00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 0x7fa80c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:17.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.669+0000 7fa82fe00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 0x7fa80c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa820000f60 tx=0x7fa820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:17.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.673+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa828059a00 con 0x7fa83015e7e0 2024-04-12T19:52:17.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.875+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 --> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8300755c0 con 0x7fa80c06c8f0 2024-04-12T19:52:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.876+0000 7fa827e00640 1 -- 172.21.15.32:0/3539393227 <== mgr.4101 v2:172.21.15.32:6832/1274114417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13395 (secure 0 0 0) 0x7fa8300755c0 con 0x7fa80c06c8f0 2024-04-12T19:52:17.877 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:17.877 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-12T19:52:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 msgr2=0x7fa80c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 0x7fa80c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa820000f60 tx=0x7fa820009040 comp rx=0 tx=0).stop 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 msgr2=0x7fa8301f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b410 tx=0x7fa82800b8d0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 shutdown_connections 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7fa80c06c8f0 0x7fa80c06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa83015e7e0 0x7fa8301f5180 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8300b6ea0 0x7fa8301f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 --2- 172.21.15.32:0/3539393227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa83009aa40 0x7fa8301541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 >> 172.21.15.32:0/3539393227 conn(0x7fa8300b40a0 msgr2=0x7fa8300b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.879+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 shutdown_connections 2024-04-12T19:52:17.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:17.880+0000 7fa835c00640 1 -- 172.21.15.32:0/3539393227 wait complete. 2024-04-12T19:52:17.902 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-12T19:52:17.660626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241516,"kb_used_data":4552,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733204,"statfs":{"total":773094113280,"available":772846800896,"internally_reserved":0,"allocated":4661248,"data_stored":2533704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"1.0","version":"38'119","reported_seq":210,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-12T19:52:13.079577+0000","last_change":"2024-04-12T19:52:13.079577+0000","last_active":"2024-04-12T19:52:13.079577+0000","last_peered":"2024-04-12T19:52:13.079577+0000","last_clean":"2024-04-12T19:52:13.079577+0000","last_became_active":"2024-04-12T19:50:13.965432+0000","last_became_peered":"2024-04-12T19:50:13.965432+0000","last_unstale":"2024-04-12T19:52:13.079577+0000","last_undegraded":"2024-04-12T19:52:13.079577+0000","last_fullsized":"2024-04-12T19:52:13.079577+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"38'119","last_scrub_stamp":"2024-04-12T19:52:13.079301+0000","last_deep_scrub":"38'119","last_deep_scrub_stamp":"2024-04-12T19:52:13.079301+0000","last_clean_scrub_stamp":"2024-04-12T19:52:13.079301+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T04:45:07.713306+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31604,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1028096,"data_stored":759489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":169121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T19:52:17.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-12T19:52:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.107+0000 7f7cf1000640 1 Processor -- start 2024-04-12T19:52:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.108+0000 7f7cf1000640 1 -- start start 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.109+0000 7f7cf1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7cec0cebb0 0x7f7cec167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.109+0000 7f7cf1000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0cd2d0 con 0x7f7cec0725f0 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.109+0000 7f7cf1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0cd5b0 con 0x7f7cec1461c0 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.109+0000 7f7cf1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7cec0cf1d0 con 0x7f7cec0cebb0 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.110+0000 7f7ce2a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cec0725f0 0x7f7cec0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50472/0 (socket says 172.21.15.32:50472) 2024-04-12T19:52:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.110+0000 7f7ce2a00640 1 -- 172.21.15.32:0/1404282238 learned_addr learned my addr 172.21.15.32:0/1404282238 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-12T19:52:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.110+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1404282238 >> v2:172.21.15.169:3300/0 conn(0x7f7cec0cebb0 0x7f7cec167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.110+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846949007 0 0) 0x7f7cec0cd2d0 con 0x7f7cec0725f0 2024-04-12T19:52:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.110+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd0003520 con 0x7f7cec0725f0 2024-04-12T19:52:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109123879 0 0) 0x7f7cd0003520 con 0x7f7cec0725f0 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 >> v2:172.21.15.169:3300/0 conn(0x7f7cec0cebb0 msgr2=0x7f7cec167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 --2- 172.21.15.32:0/1404282238 >> v2:172.21.15.169:3300/0 conn(0x7f7cec0cebb0 0x7f7cec167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 >> v1:172.21.15.169:6789/0 conn(0x7f7cec1461c0 legacy=0x7f7cec071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec0cd890 con 0x7f7cec0725f0 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1925929607 0 0) 0x7f7cd4001670 con 0x7f7cec0725f0 2024-04-12T19:52:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 >> v1:172.21.15.32:6789/0 conn(0x7f7cec0725f0 legacy=0x7f7cec0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cd0003a00 0x7f7cd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd0004800 0x7f7cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 0x7f7cd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000c090 con 0x7f7cd00051c0 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000c240 con 0x7f7cd0004800 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd000c440 con 0x7f7cd0003a00 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.111+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 0x7f7cd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7ce2a00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd0004800 0x7f7cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7ceb400640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cd0003a00 0x7f7cd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7cebe00640 1 -- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cd0003a00 msgr2=0x7f7cd0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cd0003a00 0x7f7cd0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7cebe00640 1 -- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd0004800 msgr2=0x7f7cd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd0004800 0x7f7cd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7cebe00640 1 -- 172.21.15.32:0/1404282238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0003520 con 0x7f7cd00051c0 2024-04-12T19:52:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.112+0000 7f7ceb400640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cd0003a00 0x7f7cd0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.113+0000 7f7cebe00640 1 -- 172.21.15.32:0/1404282238 reap_dead start 2024-04-12T19:52:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.113+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 0x7f7cd000b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ac0 tx=0x7f7cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34abbe931c5e529 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.113+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4009690 con 0x7f7cd00051c0 2024-04-12T19:52:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.113+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004020 con 0x7f7cd00051c0 2024-04-12T19:52:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.113+0000 7f7ceaa00640 1 -- 172.21.15.32:0/1404282238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004410 con 0x7f7cd00051c0 2024-04-12T19:52:18.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 -- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 msgr2=0x7f7cd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 0x7f7cd000b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ac0 tx=0x7f7cd0005830 comp rx=0 tx=0).stop 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 -- 172.21.15.32:0/1404282238 shutdown_connections 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1404282238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cd00051c0 0x7f7cd000b9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 -- 172.21.15.32:0/1404282238 >> 172.21.15.32:0/1404282238 conn(0x7f7cec06b930 msgr2=0x7f7cec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 -- 172.21.15.32:0/1404282238 shutdown_connections 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.114+0000 7f7cf1000640 1 -- 172.21.15.32:0/1404282238 wait complete. 2024-04-12T19:52:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.116+0000 7f7cf1000640 1 Processor -- start 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.116+0000 7f7cf1000640 1 -- start start 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd00051c0 0x7f7cec1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cec1402a0 0x7f7cec1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec0ce4a0 con 0x7f7cec142330 2024-04-12T19:52:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec0cd080 con 0x7f7cd00051c0 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.117+0000 7f7cf1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec0ce620 con 0x7f7cec1402a0 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48784/0 (socket says 172.21.15.32:48784) 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 -- 172.21.15.32:0/1304628707 learned_addr learned my addr 172.21.15.32:0/1304628707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7ceb400640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd00051c0 0x7f7cec1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7ce2a00640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cec1402a0 0x7f7cec1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 -- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cec1402a0 msgr2=0x7f7cec1406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cec1402a0 0x7f7cec1406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 -- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd00051c0 msgr2=0x7f7cec1f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd00051c0 0x7f7cec1f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 -- 172.21.15.32:0/1304628707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec143000 con 0x7f7cec142330 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.118+0000 7f7cebe00640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009bb0 tx=0x7f7ce4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.119+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4009690 con 0x7f7cec142330 2024-04-12T19:52:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.119+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce402cc50 con 0x7f7cec142330 2024-04-12T19:52:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.119+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4003610 con 0x7f7cec142330 2024-04-12T19:52:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.119+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1432e0 con 0x7f7cec142330 2024-04-12T19:52:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.119+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1438a0 con 0x7f7cec142330 2024-04-12T19:52:18.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.120+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec0755c0 con 0x7f7cec142330 2024-04-12T19:52:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.121+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ce40037b0 con 0x7f7cec142330 2024-04-12T19:52:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.122+0000 7f7ce3e00640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 0x7f7cc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.122+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f7ce40ac5e0 con 0x7f7cec142330 2024-04-12T19:52:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.122+0000 7f7ceb400640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 0x7f7cc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.123+0000 7f7ceb400640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 0x7f7cc806ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc001e40 tx=0x7f7cdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.127+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ce407ac70 con 0x7f7cec142330 2024-04-12T19:52:18.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.331+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7cec072e40 con 0x7f7cec142330 2024-04-12T19:52:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.337+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f7ce407a570 con 0x7f7cec142330 2024-04-12T19:52:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.338+0000 7f7ce3e00640 1 -- 172.21.15.32:0/1304628707 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7ce4004020 con 0x7f7cec142330 2024-04-12T19:52:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 msgr2=0x7f7cc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 0x7f7cc806ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc001e40 tx=0x7f7cdc009040 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 msgr2=0x7f7cec1427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009bb0 tx=0x7f7ce4004730 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 shutdown_connections 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7f7cc806c920 0x7f7cc806ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cec142330 0x7f7cec1427e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cec1402a0 0x7f7cec1406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 --2- 172.21.15.32:0/1304628707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cd00051c0 0x7f7cec1f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 >> 172.21.15.32:0/1304628707 conn(0x7f7cec06b930 msgr2=0x7f7cec141060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 shutdown_connections 2024-04-12T19:52:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.340+0000 7f7cf1000640 1 -- 172.21.15.32:0/1304628707 wait complete. 2024-04-12T19:52:18.359 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-12T19:52:18.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T19:52:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.576+0000 7efdc3e00640 1 Processor -- start 2024-04-12T19:52:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.576+0000 7efdc3e00640 1 -- start start 2024-04-12T19:52:18.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdc4148130 0x7efdc415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc4071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdc4072490 0x7efdc406dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efdc4148510 con 0x7efdc4072490 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efdc415fd70 con 0x7efdc4148130 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdc4162940 con 0x7efdc4162380 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.577+0000 7efdc3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdc4072490 0x7efdc406dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdc4072490 0x7efdc406dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48798/0 (socket says 172.21.15.32:48798) 2024-04-12T19:52:18.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 -- 172.21.15.32:0/3079058152 learned_addr learned my addr 172.21.15.32:0/3079058152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 -- 172.21.15.32:0/3079058152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 msgr2=0x7efdc4071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 --2- 172.21.15.32:0/3079058152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc4071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 -- 172.21.15.32:0/3079058152 >> v2:172.21.15.169:3300/0 conn(0x7efdc4148130 msgr2=0x7efdc415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 --2- 172.21.15.32:0/3079058152 >> v2:172.21.15.169:3300/0 conn(0x7efdc4148130 0x7efdc415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.578+0000 7efdc3400640 1 -- 172.21.15.32:0/3079058152 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc4162c20 con 0x7efdc4072490 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.579+0000 7efdc3400640 1 --2- 172.21.15.32:0/3079058152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4072490 0x7efdc406dcd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efdc4073010 tx=0x7efdbc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a78ee1ca8e52d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.579+0000 7efdc1600640 1 -- 172.21.15.32:0/3079058152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc02ca80 con 0x7efdc4072490 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.579+0000 7efdc1600640 1 -- 172.21.15.32:0/3079058152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efdbc02cc20 con 0x7efdc4072490 2024-04-12T19:52:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.579+0000 7efdc1600640 1 -- 172.21.15.32:0/3079058152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0343f0 con 0x7efdc4072490 2024-04-12T19:52:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 -- 172.21.15.32:0/3079058152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4072490 msgr2=0x7efdc406dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 --2- 172.21.15.32:0/3079058152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4072490 0x7efdc406dcd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efdc4073010 tx=0x7efdbc02bee0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 -- 172.21.15.32:0/3079058152 shutdown_connections 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 --2- 172.21.15.32:0/3079058152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4072490 0x7efdc406dcd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 --2- 172.21.15.32:0/3079058152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc4071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 --2- 172.21.15.32:0/3079058152 >> v2:172.21.15.169:3300/0 conn(0x7efdc4148130 0x7efdc415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 -- 172.21.15.32:0/3079058152 >> 172.21.15.32:0/3079058152 conn(0x7efdc406b930 msgr2=0x7efdc406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 -- 172.21.15.32:0/3079058152 shutdown_connections 2024-04-12T19:52:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.580+0000 7efdc3e00640 1 -- 172.21.15.32:0/3079058152 wait complete. 2024-04-12T19:52:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.582+0000 7efdc3e00640 1 Processor -- start 2024-04-12T19:52:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.582+0000 7efdc3e00640 1 -- start start 2024-04-12T19:52:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 0x7efdc41421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc41f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efdc40aeb90 con 0x7efdc4148130 2024-04-12T19:52:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41631e0 con 0x7efdc4072490 2024-04-12T19:52:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdc40aed10 con 0x7efdc4162380 2024-04-12T19:52:18.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.583+0000 7efdc2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48808/0 (socket says 172.21.15.32:48808) 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 -- 172.21.15.32:0/1476491749 learned_addr learned my addr 172.21.15.32:0/1476491749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2a00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 0x7efdc41421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc3400640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc41f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 -- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 msgr2=0x7efdc41f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc41f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 -- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 msgr2=0x7efdc41421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 0x7efdc41421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 -- 172.21.15.32:0/1476491749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc40b4910 con 0x7efdc4148130 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2a00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 0x7efdc41421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.584+0000 7efdc2000640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b410 tx=0x7efdb000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.585+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0015020 con 0x7efdc4148130 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.585+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efdb0011850 con 0x7efdc4148130 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.585+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0011c60 con 0x7efdc4148130 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.585+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc40b4bf0 con 0x7efdc4148130 2024-04-12T19:52:18.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.585+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc40b51b0 con 0x7efdc4148130 2024-04-12T19:52:18.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.586+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc40755c0 con 0x7efdc4148130 2024-04-12T19:52:18.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.587+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb0018020 con 0x7efdc4148130 2024-04-12T19:52:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.588+0000 7efdbaa00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T19:52:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.588+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7efdb008b8a0 con 0x7efdc4148130 2024-04-12T19:52:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.588+0000 7efdc2a00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T19:52:18.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.589+0000 7efdc2a00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 0x7efda406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006fd0 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T19:52:18.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.592+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdb001f020 con 0x7efdc4148130 2024-04-12T19:52:18.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.836+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdc415fb50 con 0x7efdc4148130 2024-04-12T19:52:18.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.837+0000 7efdbaa00640 1 -- 172.21.15.32:0/1476491749 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3872 (secure 0 0 0) 0x7efdb00172c0 con 0x7efdc4148130 2024-04-12T19:52:18.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-12T19:52:18.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/4088638334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4088638334},{"type":"v1","addr":"172.21.15.169:6834","nonce":4088638334}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2837293763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2837293763},{"type":"v1","addr":"172.21.15.32:6837","nonce":2837293763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2268541458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2268541458},{"type":"v1","addr":"172.21.15.169:6835","nonce":2268541458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5406,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/528568138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":528568138},{"type":"v1","addr":"172.21.15.32:6836","nonce":528568138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-12T19:52:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-04-12T19:52:18.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 msgr2=0x7efda406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 0x7efda406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006fd0 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 msgr2=0x7efdc415b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efdb000b410 tx=0x7efdb000b8d0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 shutdown_connections 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:6832/1274114417,v1:172.21.15.32:6833/1274114417] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdc4162380 0x7efdc41f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc4148130 0x7efdc415b7c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 --2- 172.21.15.32:0/1476491749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdc4072490 0x7efdc41421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T19:52:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 >> 172.21.15.32:0/1476491749 conn(0x7efdc406b930 msgr2=0x7efdc415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T19:52:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 shutdown_connections 2024-04-12T19:52:18.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-12T19:52:18.840+0000 7efdc3e00640 1 -- 172.21.15.32:0/1476491749 wait complete. 2024-04-12T19:52:18.861 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-12T19:52:18.862 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-12T19:52:18.862 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-12T19:52:18.862 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:18.882 INFO:tasks.ceph.mds.a:Stopped 2024-04-12T19:52:18.883 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-12T19:52:18.883 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:18.951 INFO:tasks.ceph.mds.c:Stopped 2024-04-12T19:52:18.951 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-12T19:52:18.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:18.979 INFO:tasks.ceph.mds.b:Stopped 2024-04-12T19:52:18.979 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-12T19:52:18.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.049 INFO:tasks.ceph.mds.d:Stopped 2024-04-12T19:52:19.050 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-12T19:52:19.050 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-12T19:52:19.050 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.113 INFO:tasks.ceph.osd.0:Stopped 2024-04-12T19:52:19.113 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-12T19:52:19.113 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.198 INFO:tasks.ceph.osd.1:Stopped 2024-04-12T19:52:19.199 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-12T19:52:19.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.269 INFO:tasks.ceph.osd.2:Stopped 2024-04-12T19:52:19.269 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-12T19:52:19.269 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.342 INFO:tasks.ceph.osd.3:Stopped 2024-04-12T19:52:19.343 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-12T19:52:19.343 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.395 INFO:tasks.ceph.osd.4:Stopped 2024-04-12T19:52:19.395 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-12T19:52:19.395 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.452 INFO:tasks.ceph.osd.5:Stopped 2024-04-12T19:52:19.452 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-12T19:52:19.452 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.525 INFO:tasks.ceph.osd.6:Stopped 2024-04-12T19:52:19.525 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-12T19:52:19.525 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.591 INFO:tasks.ceph.osd.7:Stopped 2024-04-12T19:52:19.591 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-12T19:52:19.591 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-12T19:52:19.591 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.679 INFO:tasks.ceph.mgr.y:Stopped 2024-04-12T19:52:19.680 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-12T19:52:19.680 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.759 INFO:tasks.ceph.mgr.x:Stopped 2024-04-12T19:52:19.759 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-12T19:52:19.759 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-12T19:52:19.759 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.808 INFO:tasks.ceph.mon.a:Stopped 2024-04-12T19:52:19.809 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-12T19:52:19.809 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.840 INFO:tasks.ceph.mon.b:Stopped 2024-04-12T19:52:19.841 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-12T19:52:19.841 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T19:52:19.927 INFO:tasks.ceph.mon.c:Stopped 2024-04-12T19:52:19.927 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-12T19:52:19.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-12T19:52:20.046 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:52:20.046 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-12T19:52:20.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:52:20.392 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-12T19:52:20.553 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:52:20.553 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-12T19:52:20.705 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:52:20.705 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-12T19:52:20.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:52:20.884 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-12T19:52:21.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:52:21.220 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-12T19:52:21.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:52:21.363 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-12T19:52:21.501 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:52:21.502 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-12T19:52:21.706 INFO:tasks.ceph:Archiving mon data... 2024-04-12T19:52:21.708 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/data/mon.a.tgz 2024-04-12T19:52:21.708 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-12T19:52:21.768 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.1UlbCFAz7H 2024-04-12T19:52:21.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1UlbCFAz7H 2024-04-12T19:52:21.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.1UlbCFAz7H 2024-04-12T19:52:22.104 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.1UlbCFAz7H is 291KB 2024-04-12T19:52:22.217 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.1UlbCFAz7H 2024-04-12T19:52:22.277 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/data/mon.b.tgz 2024-04-12T19:52:22.277 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-12T19:52:22.335 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.V22LWqwjqP 2024-04-12T19:52:22.336 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.V22LWqwjqP 2024-04-12T19:52:22.529 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.V22LWqwjqP 2024-04-12T19:52:22.685 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.V22LWqwjqP is 388KB 2024-04-12T19:52:22.794 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.V22LWqwjqP 2024-04-12T19:52:22.849 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/data/mon.c.tgz 2024-04-12T19:52:22.849 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-12T19:52:22.902 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.hKnYkdYz9k 2024-04-12T19:52:22.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.hKnYkdYz9k 2024-04-12T19:52:23.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.hKnYkdYz9k 2024-04-12T19:52:23.214 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.hKnYkdYz9k is 388KB 2024-04-12T19:52:23.324 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.hKnYkdYz9k 2024-04-12T19:52:23.381 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-12T19:52:23.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-12T19:52:23.390 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-12T19:52:23.553 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-12T19:52:23.611 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-12T19:52:23.611 INFO:tasks.ceph:Archiving crash dumps... 2024-04-12T19:52:23.613 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi032/crash 2024-04-12T19:52:23.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-12T19:52:23.714 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi169/crash 2024-04-12T19:52:23.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-12T19:52:23.810 INFO:tasks.ceph:Shutting down logrotate 2024-04-12T19:52:23.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:23.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:23.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:24.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-12T19:52:24.144 INFO:tasks.ceph:Compressing logs... 2024-04-12T19:52:24.146 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-12T19:52:24.154 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-12T19:52:24.248 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-12T19:52:24.248 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49121.log 2024-04-12T19:52:24.249 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42299.log 2024-04-12T19:52:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49037.log 2024-04-12T19:52:24.250 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37028.log 2024-04-12T19:52:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47296.log 2024-04-12T19:52:24.251 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49121.log.gz 2024-04-12T19:52:24.251 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.42299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42299.log.gz 2024-04-12T19:52:24.251 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37028.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37028.log.gz 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49037.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-04-12T19:52:24.253 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47296.log.gz 2024-04-12T19:52:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50243.log 2024-04-12T19:52:24.255 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-04-12T19:52:24.255 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-12T19:52:24.257 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.36988.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-12T19:52:24.257 INFO:teuthology.orchestra.run.smithi169.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-04-12T19:52:24.257 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42713.log 2024-04-12T19:52:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44470.log 2024-04-12T19:52:24.258 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-04-12T19:52:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-04-12T19:52:24.258 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50243.log.gz 2024-04-12T19:52:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.44470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44470.log.gz 2024-04-12T19:52:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42931.log 2024-04-12T19:52:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43015.log 2024-04-12T19:52:24.260 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-04-12T19:52:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-04-12T19:52:24.260 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42931.log.gz 2024-04-12T19:52:24.261 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-04-12T19:52:24.261 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.3.42713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42713.log.gz 2024-04-12T19:52:24.261 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37068.log 2024-04-12T19:52:24.261 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-12T19:52:24.263 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37108.log: /var/log/ceph/ceph-client.admin.37068.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-04-12T19:52:24.263 INFO:teuthology.orchestra.run.smithi169.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37068.log.gz 2024-04-12T19:52:24.263 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-12T19:52:24.263 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42318.log 2024-04-12T19:52:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49296.log 2024-04-12T19:52:24.264 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43015.log.gz 2024-04-12T19:52:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48172.log 2024-04-12T19:52:24.264 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-04-12T19:52:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-04-12T19:52:24.265 INFO:teuthology.orchestra.run.smithi032.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2024-04-12T19:52:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-04-12T19:52:24.265 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48172.log.gz 2024-04-12T19:52:24.267 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36908.log 2024-04-12T19:52:24.267 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.42318.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.1.42318.log.gz -5 2024-04-12T19:52:24.267 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-12T19:52:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41414.log 2024-04-12T19:52:24.268 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-04-12T19:52:24.268 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45191.log 2024-04-12T19:52:24.268 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-04-12T19:52:24.268 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2024-04-12T19:52:24.268 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41414.log.gz 2024-04-12T19:52:24.269 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.36908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36868.log 2024-04-12T19:52:24.269 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36908.log.gz 2024-04-12T19:52:24.270 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42717.log 2024-04-12T19:52:24.270 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-04-12T19:52:24.270 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45191.log.gz 2024-04-12T19:52:24.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-04-12T19:52:24.271 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2024-04-12T19:52:24.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46880.log 87.4% 2024-04-12T19:52:24.271 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-04-12T19:52:24.272 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41145.log 2024-04-12T19:52:24.273 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.36868.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36868.log.gz 2024-04-12T19:52:24.273 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-12T19:52:24.273 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.3.42717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42717.log.gz 2024-04-12T19:52:24.273 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-12T19:52:24.274 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-04-12T19:52:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42357.log 2024-04-12T19:52:24.275 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46880.log.gz 2024-04-12T19:52:24.275 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46214.log 2024-04-12T19:52:24.275 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41145.log.gz 2024-04-12T19:52:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42357.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42357.log.gz 2024-04-12T19:52:24.276 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50601.log 2024-04-12T19:52:24.277 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-12T19:52:24.277 INFO:teuthology.orchestra.run.smithi169.stderr: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-12T19:52:24.277 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36948.log 2024-04-12T19:52:24.277 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-04-12T19:52:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51249.log 2024-04-12T19:52:24.279 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46214.log.gz 2024-04-12T19:52:24.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49888.log 2024-04-12T19:52:24.279 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50601.log.gz 2024-04-12T19:52:24.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-04-12T19:52:24.280 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51249.log.gz 2024-04-12T19:52:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49888.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49888.log.gz 2024-04-12T19:52:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46922.log 2024-04-12T19:52:24.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-04-12T19:52:24.281 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.36948.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36948.log.gz 2024-04-12T19:52:24.281 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.36826.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-04-12T19:52:24.282 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-12T19:52:24.283 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-12T19:52:24.284 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-04-12T19:52:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44981.log 2024-04-12T19:52:24.284 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46922.log.gz 2024-04-12T19:52:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46082.log 2024-04-12T19:52:24.284 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-04-12T19:52:24.285 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-04-12T19:52:24.285 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.d.log: 90.9%/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-12T19:52:24.286 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44981.log.gz 2024-04-12T19:52:24.286 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-04-12T19:52:24.286 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46082.log.gz 2024-04-12T19:52:24.287 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46298.log 2024-04-12T19:52:24.287 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-04-12T19:52:24.287 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-04-12T19:52:24.288 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-04-12T19:52:24.289 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48349.log 2024-04-12T19:52:24.289 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46298.log.gz 2024-04-12T19:52:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51165.log 2024-04-12T19:52:24.290 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-04-12T19:52:24.291 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42346.log 2024-04-12T19:52:24.291 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48349.log.gz 2024-04-12T19:52:24.292 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50787.log 2024-04-12T19:52:24.292 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51165.log.gz 2024-04-12T19:52:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44162.log 2024-04-12T19:52:24.293 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42346.log.gz 2024-04-12T19:52:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2024-04-12T19:52:24.294 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50787.log.gz 2024-04-12T19:52:24.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-04-12T19:52:24.296 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44162.log.gz 2024-04-12T19:52:24.296 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37498.log 2024-04-12T19:52:24.297 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2024-04-12T19:52:24.298 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51039.log 2024-04-12T19:52:24.298 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-04-12T19:52:24.298 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2024-04-12T19:52:24.299 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37498.log.gz 2024-04-12T19:52:24.300 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45065.log 2024-04-12T19:52:24.300 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51039.log.gz 2024-04-12T19:52:24.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46502.log 2024-04-12T19:52:24.301 INFO:teuthology.orchestra.run.smithi032.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2024-04-12T19:52:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41713.log 2024-04-12T19:52:24.302 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45065.log.gz 2024-04-12T19:52:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-04-12T19:52:24.303 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46502.log.gz 2024-04-12T19:52:24.304 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44466.log 2024-04-12T19:52:24.304 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41713.log.gz 2024-04-12T19:52:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50056.log 2024-04-12T19:52:24.305 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-04-12T19:52:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.44466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44466.log.gz 2024-04-12T19:52:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41839.log 2024-04-12T19:52:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-04-12T19:52:24.307 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50056.log.gz 2024-04-12T19:52:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-04-12T19:52:24.309 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41839.log.gz 2024-04-12T19:52:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47756.log 2024-04-12T19:52:24.310 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-04-12T19:52:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42354.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-04-12T19:52:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-04-12T19:52:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46670.log 2024-04-12T19:52:24.312 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47756.log.gz 2024-04-12T19:52:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50871.log 2024-04-12T19:52:24.313 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gz 2024-04-12T19:52:24.314 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47216.log 2024-04-12T19:52:24.314 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46670.log.gz 2024-04-12T19:52:24.315 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48391.log 2024-04-12T19:52:24.316 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-12T19:52:24.316 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50871.log.gz 2024-04-12T19:52:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-04-12T19:52:24.317 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47216.log.gz 2024-04-12T19:52:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46796.log 2024-04-12T19:52:24.318 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48391.log.gz 2024-04-12T19:52:24.318 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42889.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-12T19:52:24.319 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-04-12T19:52:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47132.log 2024-04-12T19:52:24.320 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46796.log.gz 2024-04-12T19:52:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45149.log 2024-04-12T19:52:24.322 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47132.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-12T19:52:24.322 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-04-12T19:52:24.322 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47132.log.gz 2024-04-12T19:52:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48307.log 2024-04-12T19:52:24.323 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45149.log.gz 2024-04-12T19:52:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42692.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-04-12T19:52:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2024-04-12T19:52:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-04-12T19:52:24.325 INFO:teuthology.orchestra.run.smithi032.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48307.log.gz 2024-04-12T19:52:24.326 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-04-12T19:52:24.326 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2024-04-12T19:52:24.327 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-04-12T19:52:24.327 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2024-04-12T19:52:24.328 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2024-04-12T19:52:24.329 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-04-12T19:52:24.329 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49720.log 2024-04-12T19:52:24.330 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-04-12T19:52:24.330 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41923.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.43393.log -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2024-04-12T19:52:24.330 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-12T19:52:24.331 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43309.log 2024-04-12T19:52:24.332 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49720.log.gz 2024-04-12T19:52:24.332 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46544.log 2024-04-12T19:52:24.333 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-04-12T19:52:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43309.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43309.log.gz 2024-04-12T19:52:24.334 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37238.log 2024-04-12T19:52:24.335 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-04-12T19:52:24.335 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46544.log.gz 2024-04-12T19:52:24.336 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45722.log 2024-04-12T19:52:24.336 INFO:teuthology.orchestra.run.smithi032.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.37238.log.gz 2024-04-12T19:52:24.337 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-12T19:52:24.337 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-04-12T19:52:24.338 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-04-12T19:52:24.338 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45722.log.gz 2024-04-12T19:52:24.339 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46586.log 2024-04-12T19:52:24.339 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-12T19:52:24.340 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-04-12T19:52:24.340 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-04-12T19:52:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41583.log 2024-04-12T19:52:24.342 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46586.log.gz 2024-04-12T19:52:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47672.log 2024-04-12T19:52:24.343 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-04-12T19:52:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2024-04-12T19:52:24.344 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41583.log.gz 2024-04-12T19:52:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47672.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.47672.log.gz -- 2024-04-12T19:52:24.345 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.44729.log 2024-04-12T19:52:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42091.log 2024-04-12T19:52:24.346 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz 2024-04-12T19:52:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42175.log 2024-04-12T19:52:24.348 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-04-12T19:52:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48817.log 2024-04-12T19:52:24.349 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42091.log.gz 2024-04-12T19:52:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47714.log 2024-04-12T19:52:24.350 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42175.log.gz 2024-04-12T19:52:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48817.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-12T19:52:24.351 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48817.log.gz 2024-04-12T19:52:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-04-12T19:52:24.352 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47714.log.gz 2024-04-12T19:52:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-04-12T19:52:24.354 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-04-12T19:52:24.354 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2024-04-12T19:52:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50517.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-04-12T19:52:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2024-04-12T19:52:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45233.log 2024-04-12T19:52:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42682.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-04-12T19:52:24.357 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2024-04-12T19:52:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50661.log 2024-04-12T19:52:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47546.log 2024-04-12T19:52:24.359 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45233.log.gz 2024-04-12T19:52:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50661.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50661.log.gz 2024-04-12T19:52:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50997.log 2024-04-12T19:52:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42349.log 2024-04-12T19:52:24.362 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1%/var/log/ceph/ceph-client.admin.50997.log: -- replaced with /var/log/ceph/ceph-client.admin.47546.log.gz 2024-04-12T19:52:24.362 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50997.log.gz 2024-04-12T19:52:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49972.log 2024-04-12T19:52:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50913.log 2024-04-12T19:52:24.364 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42349.log.gz 2024-04-12T19:52:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49972.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49972.log.gz 2024-04-12T19:52:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44010.log 2024-04-12T19:52:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37668.log 2024-04-12T19:52:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.44010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50913.log.gz 2024-04-12T19:52:24.366 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44010.log.gz 2024-04-12T19:52:24.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-04-12T19:52:24.368 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-04-12T19:52:24.368 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37668.log.gz 2024-04-12T19:52:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42049.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-04-12T19:52:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41456.log 2024-04-12T19:52:24.370 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-12T19:52:24.371 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-04-12T19:52:24.371 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41456.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41456.log.gz 2024-04-12T19:52:24.371 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-04-12T19:52:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-04-12T19:52:24.373 INFO:teuthology.orchestra.run.smithi169.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-12T19:52:24.374 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-12T19:52:24.374 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46712.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-04-12T19:52:24.374 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48813.log 2024-04-12T19:52:24.375 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41499.log 2024-04-12T19:52:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48813.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-04-12T19:52:24.376 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48813.log.gz 2024-04-12T19:52:24.376 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-04-12T19:52:24.377 INFO:teuthology.orchestra.run.smithi169.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-12T19:52:24.377 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41499.log: -- /var/log/ceph/ceph-client.admin.45107.log 2024-04-12T19:52:24.378 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42007.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41499.log.gz 2024-04-12T19:52:24.379 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-04-12T19:52:24.379 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46040.log 2024-04-12T19:52:24.379 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-12T19:52:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51081.log 2024-04-12T19:52:24.380 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45107.log.gz 2024-04-12T19:52:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46040.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46040.log.gz 2024-04-12T19:52:24.381 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49259.log 2024-04-12T19:52:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41755.log 2024-04-12T19:52:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49259.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51081.log.gz 2024-04-12T19:52:24.383 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49259.log.gz 2024-04-12T19:52:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42688.log 2024-04-12T19:52:24.384 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47006.log 2024-04-12T19:52:24.385 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41755.log.gz 2024-04-12T19:52:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42688.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42688.log.gz 2024-04-12T19:52:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47504.log 2024-04-12T19:52:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42677.log 2024-04-12T19:52:24.387 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47006.log.gz 2024-04-12T19:52:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47504.log.gz 2024-04-12T19:52:24.388 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-12T19:52:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48559.log 2024-04-12T19:52:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42677.log: 87.8%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.42677.log.gz 2024-04-12T19:52:24.390 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45764.log 2024-04-12T19:52:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48517.log 2024-04-12T19:52:24.393 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48559.log.gz 2024-04-12T19:52:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45275.log 2024-04-12T19:52:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45764.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45764.log.gz 2024-04-12T19:52:24.394 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-04-12T19:52:24.395 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48517.log.gz 2024-04-12T19:52:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50955.log 2024-04-12T19:52:24.396 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2024-04-12T19:52:24.397 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-04-12T19:52:24.398 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-04-12T19:52:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-04-12T19:52:24.399 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50955.log.gz 2024-04-12T19:52:24.400 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49678.log 2024-04-12T19:52:24.400 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-04-12T19:52:24.401 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2024-04-12T19:52:24.401 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-04-12T19:52:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-04-12T19:52:24.403 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49678.log.gz 2024-04-12T19:52:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-04-12T19:52:24.404 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2024-04-12T19:52:24.404 INFO:teuthology.orchestra.run.smithi169.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-12T19:52:24.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-04-12T19:52:24.405 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-04-12T19:52:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-04-12T19:52:24.406 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-04-12T19:52:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-04-12T19:52:24.408 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-04-12T19:52:24.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-04-12T19:52:24.409 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2024-04-12T19:52:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-04-12T19:52:24.410 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-04-12T19:52:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46628.log 2024-04-12T19:52:24.411 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-04-12T19:52:24.412 INFO:teuthology.orchestra.run.smithi169.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-12T19:52:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42356.log 2024-04-12T19:52:24.413 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-04-12T19:52:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47588.log 2024-04-12T19:52:24.414 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46628.log.gz 2024-04-12T19:52:24.415 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42356.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-12T19:52:24.415 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-12T19:52:24.416 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42356.log.gz 2024-04-12T19:52:24.416 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46340.log 2024-04-12T19:52:24.416 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47588.log.gz 2024-04-12T19:52:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-04-12T19:52:24.418 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-12T19:52:24.419 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m0.211s 2024-04-12T19:52:24.419 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m0.794s 2024-04-12T19:52:24.419 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.201s 2024-04-12T19:52:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-04-12T19:52:24.422 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46340.log.gz 2024-04-12T19:52:24.422 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47418.log 2024-04-12T19:52:24.422 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-04-12T19:52:24.423 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48625.log 2024-04-12T19:52:24.423 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-04-12T19:52:24.424 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47418.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-04-12T19:52:24.425 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47418.log.gz 2024-04-12T19:52:24.425 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-04-12T19:52:24.425 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-04-12T19:52:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46838.log 2024-04-12T19:52:24.426 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-04-12T19:52:24.427 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-04-12T19:52:24.427 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-04-12T19:52:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-04-12T19:52:24.428 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46838.log.gz 2024-04-12T19:52:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44813.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-04-12T19:52:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50159.log 2024-04-12T19:52:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46256.log 2024-04-12T19:52:24.431 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz 2024-04-12T19:52:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50159.log.gz 2024-04-12T19:52:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48995.log 2024-04-12T19:52:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-04-12T19:52:24.433 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46256.log.gz 2024-04-12T19:52:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2024-04-12T19:52:24.434 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48995.log.gz 2024-04-12T19:52:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49395.log 2024-04-12T19:52:24.435 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-04-12T19:52:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-04-12T19:52:24.437 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2024-04-12T19:52:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-04-12T19:52:24.438 INFO:teuthology.orchestra.run.smithi032.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.49395.log.gz 2024-04-12T19:52:24.439 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-04-12T19:52:24.439 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-04-12T19:52:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-12T19:52:24.440 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-04-12T19:52:24.441 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45359.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz -5 2024-04-12T19:52:24.441 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46124.log 2024-04-12T19:52:24.442 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48220.log 2024-04-12T19:52:24.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-04-12T19:52:24.444 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46124.log.gz 2024-04-12T19:52:24.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41881.log 2024-04-12T19:52:24.445 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48220.log.gz 2024-04-12T19:52:24.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50014.log 2024-04-12T19:52:24.447 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-04-12T19:52:24.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41881.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41881.log.gz 2024-04-12T19:52:24.448 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.42344.log 2024-04-12T19:52:24.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50014.log.gz 2024-04-12T19:52:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48025.log 2024-04-12T19:52:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-04-12T19:52:24.451 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42344.log.gz 2024-04-12T19:52:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48025.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48025.log.gz 2024-04-12T19:52:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-04-12T19:52:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2024-04-12T19:52:24.454 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-04-12T19:52:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48128.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-04-12T19:52:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45956.log 2024-04-12T19:52:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44603.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-04-12T19:52:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-12T19:52:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-04-12T19:52:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45956.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45956.log.gz 2024-04-12T19:52:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47462.log 2024-04-12T19:52:24.460 INFO:teuthology.orchestra.run.smithi032.stderr: 84.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-12T19:52:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gz 2024-04-12T19:52:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41625.log 2024-04-12T19:52:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-04-12T19:52:24.462 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47462.log.gz 2024-04-12T19:52:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-04-12T19:52:24.463 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41625.log.gz 2024-04-12T19:52:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42691.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-04-12T19:52:24.465 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-12T19:52:24.466 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48067.log 2024-04-12T19:52:24.466 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-04-12T19:52:24.467 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-12T19:52:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48067.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48067.log.gz 2024-04-12T19:52:24.469 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-12T19:52:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2024-04-12T19:52:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2024-04-12T19:52:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2024-04-12T19:52:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31223.log 2024-04-12T19:52:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45998.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2024-04-12T19:52:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49352.log 2024-04-12T19:52:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.31223.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31223.log.gz 2024-04-12T19:52:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46460.log 2024-04-12T19:52:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49352.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49352.log.gz 2024-04-12T19:52:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46460.log: -- /var/log/ceph/ceph-client.admin.43099.log 2024-04-12T19:52:24.479 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46460.log.gz 2024-04-12T19:52:24.480 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-04-12T19:52:24.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43099.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-12T19:52:24.483 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log/var/log/ceph/ceph-client.admin.43537.log: 2024-04-12T19:52:24.484 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-04-12T19:52:24.485 INFO:teuthology.orchestra.run.smithi032.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-12T19:52:24.485 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47798.log 2024-04-12T19:52:24.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43225.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-04-12T19:52:24.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47798.log.gz 2024-04-12T19:52:24.488 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44687.log 2024-04-12T19:52:24.489 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2024-04-12T19:52:24.491 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51415.log 2024-04-12T19:52:24.491 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44687.log.gz 2024-04-12T19:52:24.492 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49804.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2024-04-12T19:52:24.493 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51415.log.gz 2024-04-12T19:52:24.494 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-04-12T19:52:24.495 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49636.log 2024-04-12T19:52:24.497 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-04-12T19:52:24.497 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-04-12T19:52:24.498 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49636.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49636.log.gz 2024-04-12T19:52:24.499 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-04-12T19:52:24.500 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-04-12T19:52:24.501 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46964.log 2024-04-12T19:52:24.503 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50829.log 2024-04-12T19:52:24.503 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-04-12T19:52:24.504 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46964.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46964.log.gz 2024-04-12T19:52:24.505 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50829.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50829.log.gz 2024-04-12T19:52:24.506 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50285.log 2024-04-12T19:52:24.507 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-04-12T19:52:24.509 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2024-04-12T19:52:24.509 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50285.log.gz 2024-04-12T19:52:24.510 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47630.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-04-12T19:52:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49762.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2024-04-12T19:52:24.512 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49079.log 2024-04-12T19:52:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-04-12T19:52:24.513 INFO:teuthology.orchestra.run.smithi032.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-12T19:52:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49079.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-12T19:52:24.515 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49079.log.gz 2024-04-12T19:52:24.517 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44029.log 2024-04-12T19:52:24.517 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-04-12T19:52:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2024-04-12T19:52:24.520 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.44029.log: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-12T19:52:24.520 INFO:teuthology.orchestra.run.smithi032.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41237.log 2024-04-12T19:52:24.520 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.0.44029.log.gz 2024-04-12T19:52:24.521 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51207.log 2024-04-12T19:52:24.521 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2024-04-12T19:52:24.523 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41669.log 2024-04-12T19:52:24.524 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41237.log.gz 2024-04-12T19:52:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42133.log 2024-04-12T19:52:24.524 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51207.log.gz 2024-04-12T19:52:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41669.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41669.log.gz 2024-04-12T19:52:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42133.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42133.log.gz 2024-04-12T19:52:24.528 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-12T19:52:24.553 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-12T19:52:24.591 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-12T19:52:24.604 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-12T19:52:24.609 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-12T19:52:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m0.406s 2024-04-12T19:52:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m0.973s 2024-04-12T19:52:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.725s 2024-04-12T19:52:24.610 INFO:tasks.ceph:Archiving logs... 2024-04-12T19:52:24.611 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi032/log 2024-04-12T19:52:24.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T19:52:25.023 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi169/log 2024-04-12T19:52:25.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T19:52:25.230 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-12T19:52:25.254 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-12T19:52:25.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T19:52:25.261 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T19:52:25.410 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-12T19:52:25.419 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-12T19:52:25.419 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T19:52:25.428 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T19:52:25.478 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-04-12T19:52:25.481 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-12T19:52:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T19:52:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-04-12T19:52:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:^- hv01.front.sepia.ceph.com 2 6 377 0 +1221us[+1221us] +/- 1033ms 2024-04-12T19:52:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -458us[ -463us] +/- 63ms 2024-04-12T19:52:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +651us[ +651us] +/- 86ms 2024-04-12T19:52:25.490 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +1216us[+1216us] +/- 1033ms 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -457us[ -457us] +/- 63ms 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +648us[ +648us] +/- 86ms 2024-04-12T19:52:25.491 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T19:52:25.492 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-12T19:52:25.502 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-12T19:52:25.502 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-12T19:52:25.511 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T19:52:25.749 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T19:52:25.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T19:52:26.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:26.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T19:52:26.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:26.177 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-12T19:52:26.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-12T19:52:26.197 INFO:teuthology.task.internal:Duration was 730.358566 seconds 2024-04-12T19:52:26.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-12T19:52:26.207 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-12T19:52:26.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T19:52:26.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T19:52:26.371 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T19:52:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T19:52:26.839 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-12T19:52:26.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-04-12T19:52:26.840 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T19:52:26.903 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-12T19:52:26.903 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T19:52:26.980 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-12T19:52:26.980 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T19:52:26.989 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T19:52:27.085 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-12T19:52:27.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T19:52:27.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T19:52:27.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-12T19:52:27.288 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-12T19:52:27.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T19:52:27.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T19:52:27.487 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-12T19:52:27.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T19:52:27.505 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T19:52:27.616 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-04-12T19:52:27.616 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-12T19:52:27.673 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T19:52:27.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:27.726 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T19:52:27.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T19:52:27.783 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-12T19:52:27.803 INFO:teuthology.task.internal:Transferring archived files... 2024-04-12T19:52:27.804 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi032 2024-04-12T19:52:27.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T19:52:27.919 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533/remote/smithi169 2024-04-12T19:52:27.920 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T19:52:28.026 INFO:teuthology.task.internal:Removing archive directory... 2024-04-12T19:52:28.027 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T19:52:28.036 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T19:52:28.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-12T19:52:28.106 INFO:teuthology.task.internal:Not uploading archives. 2024-04-12T19:52:28.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-12T19:52:28.121 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-12T19:52:28.121 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T19:52:28.129 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T19:52:28.178 INFO:teuthology.orchestra.run.smithi032.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 12 19:52 /home/ubuntu/cephtest 2024-04-12T19:52:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 12 19:52 /home/ubuntu/cephtest 2024-04-12T19:52:28.188 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-12T19:52:28.199 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-12T19:52:28.221 INFO:teuthology.nuke:Checking targets against current locks 2024-04-12T19:52:28.247 DEBUG:teuthology.nuke:shortname: smithi032 2024-04-12T19:52:28.248 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T19:52:28.271 DEBUG:teuthology.nuke:shortname: smithi169 2024-04-12T19:52:28.272 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T19:52:28.275 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 19:29:08.044209', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDUEA8WnjKZ9paXPsDNdA3GYnTxMov/p6jCpaJpj6rVwsfiQF01MzE5aCFaFiMwm7KD0tEWLAYQRg8OjJZxcx2w='} 2024-04-12T19:52:28.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653533', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 19:29:08.042806', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCUDIFxY/GFxMBPu6HqD4URgoVn/Pp03fHbk3an3renx91zpeRxaJNCymGgVDBKijvsELAohvemX2LJoFj1EYao='} 2024-04-12T19:52:28.331 INFO:teuthology.orchestra.console.smithi032:Power off 2024-04-12T19:52:28.332 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T19:52:28.357 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-04-12T19:52:28.358 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:28.375 INFO:teuthology.orchestra.console.smithi169:Power off 2024-04-12T19:52:28.376 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T19:52:28.386 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-12T19:52:28.399 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-04-12T19:52:28.400 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:28.423 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-12T19:52:32.388 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:32.425 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:32.500 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-12T19:52:32.536 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-12T19:52:36.504 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:36.537 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:36.616 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-04-12T19:52:36.648 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-12T19:52:36.717 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-04-12T19:52:40.650 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T19:52:40.761 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-12T19:52:40.862 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-04-12T19:52:40.968 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 client.1-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-flush} duration: 730.3585658073425 failure_reason: 'Test failure: test_flush (tasks.cephfs.test_flush.TestFlush)' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=755c47263f0843efbcd7c4e919486483 status: fail success: false 2024-04-12T19:52:40.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T19:52:41.045 INFO:teuthology.run:FAIL